![](\"https:\/\/sites.bu.edu\/phenogeno\/files\/2020\/12\/IMG_7184-769x1024.jpg\")
<\/h3>\n<\/h3>\nI am sharing these highlights not to brag about accomplishments but to educate and share lessons from many exceptional mentors, students, fellows, collaborators and unusual colleagues I was fortunate to work with over the course of my career.<\/p>\n
I co-mentored more than 60 students and fellows. I collaborated directly (one on one) with over 70 scientists (not including consortium level collaborators) The full list can be found here:<\/p>\n
<\/p>\n
I became interested in AI early on mostly because of the work by Zohar Manna on “Program Synthesis”.<\/span>\u00a0 I discovered Zohar Manna’s work while taking a course from Amir Pnueli.\u00a0 Zohar’s approach to program synthesis was based on deductive reasoning.\u00a0 Brilliant work but I felt deduction can (and should be) integrated with induction (learning).\u00a0<\/strong><\/p>\n I brought my own ideas into this mix but the work I did with my PhD co-mentors\u00a0 Azriel and Jack was largely independent of each other with some conceptual leakage but not a real marriage.<\/p>\n I took an unusual path during my PhD because I was co-mentored by two legendary AI researchers who were very different ideologically, philosophically and methodologically. Jack Minker<\/strong> <\/span>was following the logic foundation of AI and was focusing on Logic Programming<\/strong>. He was one of the fathers of Deductive Databases or what is sometimes called Datalog.\u00a0 Azriel Rosenfeld <\/span>was one of the fathers of Computer Vision focusing on Perception, Cognition, Computer Vision, Constraints, Neuronal and Pyramidal Networks. However they shared one important thing in common: a BIG heart and empathy at a rare scale.\u00a0<\/span><\/strong><\/p>\n This was an extremely complex implementation since the machine had NO operating system and we had to implement all messaging and parallel execution on our own.\u00a0 Many highly capable people were involved in this project but much credit goes to Madhur Kohli (a very gifted systems researcher)\u00a0 for carrying the heaviest programming on his shoulders and Jack of course for his early vision.<\/p>\n <\/p>\n In 1985 AI was experiencing explosive growth.\u00a0 The timing was fortunate for anyone in AI seeking academic positions.<\/p>\n My area of parallel AI was particularly hot in part due to the 5th Generation AI Machines Project<\/strong> <\/span>in Japan that forced the US to respond.<\/p>\n But I expect that the positive response to my applications was in part driven by the generosity of my letter writers and their considerable stature. This academic joke is humorous and self deprecating version of faculty interviews.\u00a0 There is a lot of good work out there of course but the joke is still amusing and provides a perspective.<\/a><\/p>\n I visited eight universities and received seven offers straight out of PhD.<\/p>\n I had a very unusual approach to job applications (not recommended).\u00a0 We intersected nice and relatively small cities ranked high for quality of life in the Almanac with top 20 CS departments. I did not apply to any universities in major cities or places that were not ranked high in quality of life in the Almanac. We also excluded the West Coast.\u00a0 At the completion of the interviews I was mostly considering U. Wisconsin (Madison) and Duke U. But at the end, for complex personal and geographic reasons to stay in the DC area,\u00a0 I applied late to Johns Hopkins University (which was neither top 20 in CS nor in highly ranked city in the Almanac) and joined the department of EECS a week or so after my interview.\u00a0 Johns Hopkins U. is a unique institution and I met many life long friends there. I was particularly influenced by Terry Sejnowski, Fred Jelinek, Vernon Mountcastle, Ham Smith, Adi Karni (I audited his Decision Theory Class as a faculty guest), Sol Snyder (I audited his Neuroscience class as a faculty guest) and many direct collaborators and students.<\/p>\n As mentioned,\u00a0 our academic generation was very lucky.\u00a0 Five years later the market dried up, AI was experiencing a winter and it became very challenging to obtain an academic position in AI.<\/p>\n In 2020 (today) AI applicants are lucky again as deep learning is driving enormous interest and hiring in AI.<\/p>\n <\/p>\n <\/p>\n After joining Hopkins,\u00a0 I din’t want to continue with heavy software development work and implementations of parallel AI systems. I realized that parallel machines were not ready for us and the overhead of developing the operating system, messaging, and parallel execution on raw architectures were exceptionally challenging but had little todo with basic AI research. Also Hopkins was\u00a0 very small (4 CS faculty total) and unlike universities such as CMU or Illinois the group did not have opportunities for collaboration with computer systems researchers.<\/p>\n I decided to shift research interests and marry parallel algorithms and complexity with AI.<\/span><\/strong> Together with exceptional PhD students we produced several foundational results developing basic AI algorithms on abstract parallel models of computation.\u00a0 It was a niche area since parallel algorithms were largely studied by CS theorists who focused on basic graph problems or sorting not AI algorithms.\u00a0 Most of core AI people at that time worked on heuristic search or ad-hoc machine learning ignoring parallel machines (with few notable exceptions).<\/p>\n This\u00a0 allowed our group to have relatively little competition in this space and we established many basic theoretical and other textbook results that stood the test of time. Selected results are listed below and the surprising or fundamental results are highlighted. All the work was theoretical (except the amazing work by Lewis Stiller on the Connection Machine 2).<\/p>\n I was very fortunate working with exceptional PhD students in Parallel AI who walked on water. In particular, Art Delcher<\/strong><\/span> became a life long friend and collaborator. Lewis Stiller<\/span><\/strong> did exceptionally challenging research on chess end game analysis on a 65000 processor parallel machine (Thinking Machines, CM2). He used techniques from computer algebra and group theory integrated with AI and parallel software development.<\/p>\n We also had a number of very gifted undergraduate students rotating with us and doing projects in parallel AI. They are listed on the eco system page.<\/p>\n <\/p>\n I wanted to go back to software development and because of Lewis Stiller’s work read up on Decision Diagrams and Decision Trees. At the same time Steven Salzberg joined Hopkins. Steven was an applied machine learning researcher who was interested in developing and validating ML software. Our collaboration proved very fruitful because I was interested in theoretical and conceptually novel developments and Steven cared deeply about dissemination, testing and validation.<\/p>\n We collaborated both on systems development and novel theoretical ideas.<\/p>\n The main topics are highlighted below.<\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n","protected":false},"excerpt":{"rendered":" My Research Journey (in construction) I am sharing these highlights not to brag about accomplishments but to educate and share lessons from many exceptional mentors, students, fellows, collaborators and unusual colleagues I was fortunate to work with over the course of my career. Because of my unusual style of collaboration, the story also tracks several […]<\/p>\n","protected":false},"author":8782,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":[],"_links":{"self":[{"href":"https:\/\/sites.bu.edu\/phenogeno\/wp-json\/wp\/v2\/pages\/2001"}],"collection":[{"href":"https:\/\/sites.bu.edu\/phenogeno\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/sites.bu.edu\/phenogeno\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/sites.bu.edu\/phenogeno\/wp-json\/wp\/v2\/users\/8782"}],"replies":[{"embeddable":true,"href":"https:\/\/sites.bu.edu\/phenogeno\/wp-json\/wp\/v2\/comments?post=2001"}],"version-history":[{"count":50,"href":"https:\/\/sites.bu.edu\/phenogeno\/wp-json\/wp\/v2\/pages\/2001\/revisions"}],"predecessor-version":[{"id":2579,"href":"https:\/\/sites.bu.edu\/phenogeno\/wp-json\/wp\/v2\/pages\/2001\/revisions\/2579"}],"wp:attachment":[{"href":"https:\/\/sites.bu.edu\/phenogeno\/wp-json\/wp\/v2\/media?parent=2001"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}Originally, I was hoping to marry computer vision with logic<\/strong> because I thought it would be fundamental and a key to progress in general AI.<\/span><\/h3>\n
Lesson 2: <\/strong><\/span><\/h2>\n
I believe that integration of logic and perception<\/span> remains a major unsolved AI problem (40 years later).<\/strong><\/span><\/h3>\n
I learned many things from Azriel and Jack but the most important lessons were:<\/strong><\/h3>\n
Lessons 3 & 4<\/span><\/h2>\n
\n
Have an open mind.<\/span> There is more than one way to study AI (in contrast to several old AI “gurus” that treated AI as a religion rather than an expansive field similar to Biology and Physics.\u00a0 \u00a0Open AI should mean “Open minded AI”<\/span> and be inclusive.<\/h3>\n<\/li>\n
“Educating the mind<\/span> without educating the heart<\/span> is no education at all”.. Aristotle but both of my PhD mentors were subscribed to it. They routinely engaged in humanitarian and other activities outside their immediate research area.<\/h3>\n<\/li>\n<\/ol>\n
I co-authored 11+ papers<\/strong> during my PhD. All credit goes to the unusual and creative environment that Azriel and Jack had in their centers.<\/h3>\n
<\/h3>\n
Lesson 5:<\/span><\/h2>\n
For some reason that I still cannot fully understand, the connectionists \/ constraint network \/ semantic network<\/span> parts of the AI world firmly believed in parallelism (Azriel Rosenfeld was definitely a supporter of parallelism<\/span>).\u00a0 However, the main stream logic and rule based AI<\/span> scientists had little interest in it. The main lesson to young scientists. Don’t judge, just follow your passion. Our work with Jack Minker and his group included the first Parallel Logic Programming System that was actually implemented and tested on a real parallel processor (ZMOB, a 128 processors machine invented by Chuck Rieger).<\/strong> Chuck was an AI visionary who shared our belief that parallel computing is an important and perhaps even essential trait of AI systems.<\/span> This vision was later reignited by Marvin Minsky<\/span> and Danny Hillis<\/span> by building the Connection Machine (a 64000+ processor machine).<\/span><\/h3>\n
We pioneered the first implementation of fork join<\/strong> <\/span>in parallel AI. Fork-join<\/strong> <\/span>is an fundamental primitive in parallel programming. It was\u00a0 extended significantly many years later to the very famous and widely used MAP-REDUCE<\/strong><\/span> paradigm.<\/h3>\n
\n
Fast Chapter 2: Academic Jobs Search<\/h3>\n
Lesson 6:\u00a0<\/span><\/h2>\n
I am describing this experience in detail because many young PhDs take their interviews way too personally. The faculty interview success is heavily dependent on many factors that are completely independent of the candidate’s ability and accomplishments.\u00a0 So relax and enjoy the ride.\u00a0 Five years later it may change completely :)<\/strong><\/span><\/h3>\n
Chapter 3: Early Academic Research (Parallel AI)<\/h3>\n
(with Art Delcher and Lewis Stiller)<\/h3>\n
\n
Lesson 7:\u00a0<\/span><\/h2>\n
We learned many profound theoretical, technical and practical ideas, frameworks, and methods.\u00a0 However, one particular lesson that stands out is a question posed by then undergraduate research student Phoebe Sengers who was programming on the Connection Machine (64000+ processors) as an undergraduate project.\u00a0 “What would AI look like if it was pioneered and driven by women?”<\/span>\u00a0 The most influential people historically early on were Marvin Minsky, John McCarthy, Allen Newell and Herb Simon.\u00a0 Phoebe is now a professor at Cornell working at the intersection of culture and Science .<\/span><\/h3>\n
Chapter 4:\u00a0 Machine Learning: Theory and Widely Used System Development<\/h3>\n
(with Steven Salzberg, David Heath, S. Murthy, John Rachlin, David Waltz and others)<\/h3>\n
\n
\n
Lesson 8: <\/span><\/h2>\n
In my view we do not have enough long term and sustainable direct collaborations in AI between applied and theoretical research.<\/span> However, a review of the high impact literature suggests many successful examples (e.g., McCallum\/Lafferty\/Pereira, Blum\/Mitchell, Freund\/Shapire, LISP\u00a0 and many, many more).\u00a0 Many funded projects do lip service to these theory-practice collaborations but largely leave the participants in their own caves.\u00a0<\/span> It takes special people without fragile egos and ability to mutually appreciate the intellectual challenge in doing both theory and experimental research at the highest level.\u00a0 At the lowest level, theory is indeed a superior discipline at the intellectual level because it requires formal proofs at any level of the field. However, at the highest level experimental research is as magical and elegant as any theory.\u00a0 I personally remain fully committed to both.\u00a0<\/span> However, one needs to be able to selectively lower the bar on either theoretical or empirical methodologies to identify novel paths at the intersection of both. This is one of the great challenges for scientists in these philosophically and methodologically different areas.\u00a0 However, if we learn how to simultaneously advance theory and experiments, the effective translation to practice can be shortened significantly with tremendous benefits to society.\u00a0 Which is what matters the most!\u00a0<\/span><\/span><\/h3>\n
Chapter 5:\u00a0 Bayes Networks (Graphical Models) and Biology<\/h2>\n
We helped conceive and help popularize the use of Bayes networks (Graphical Models) in Biology starting 1992.<\/h3>\n
We made two predictions\u00a0 in our 1993-5 papers that provide a good motivation for the utility of graphical models in reasoning about biology or modeling biological systems.<\/span><\/h3>\n
“To summarize, scientific\f analysis of data is an important potential application of Artificial Intelligence (AI) research. We believe that the ultimate data analysis system using AI techniques will have a wide range of tools at its disposal and will adaptively choose various methods.<\/h3>\n
It will be able to generate simulations automatically and verify the model it constructed with the data generated during these simulations. When the model does not fi\ft the observed results the system will try to explain the source of error, conduct additional experiments, and choose a different model by modifying system parameters. If it needs user assistance, it will produce a simple low-dimensional view of the constructed model and the data. This will allow the user to guide the system toward constructing a new model and\/or generating the next set of experiments.”<\/h3>\n
Prediction 1: <\/span><\/h2>\n
“We believe that flexibility, efficiency and direct representation of causality in probabilistic networks are important desirable features that make them very strong candidates as a framework for biological modeling systems.”<\/span><\/h2>\n
This quote From Delcher, A., S. Kasif, H. Goldberg and W. Xsu, \u201cProtein Secondary-Structure Modeling with Probabilistic Networks\u201d, International Conference on Intelligent Systems and Molecular Biology, pp. 109\u2013117, 1993.
\n<\/span><\/h4>\nDelcher, A., S. Kasif, H. Goldberg and W. Xsu, \u201cApplication of Probabilistic Causal Trees to Analysis of Protein Secondary Structure\u201d, Proceedings of the National Conference on Artificial Intelligence, pp. 316\u2013321, July 1993.<\/span><\/h4>\n
Prediction 2: <\/span><\/h2>\n
Based on Prediction 1, we also proposed a relatively efficient computational framework to implement in-silico directed evolution (synthetic biology) using graphical models.\u00a0 More specifically, the framework must perform mutagenesis\u00a0 (e.g., amino acid substitutions in proteins) and screen for mutations and adaptations that satisfy structural and functional constraints.\u00a0 For each such perturbation (mutation) we must perform an efficient inference in the graphical model to assess the probability a given property of a protein or system was affected by the perturbation. Our novel procedure enables to COMPILE THE GRAPHICAL MODEL<\/span> using dynamic data structures and subsequently compute the change in a specific property in time that is exponentially faster than using normal algorithms.\u00a0 \u00a0This is described in detail in the paper below.\u00a0<\/span><\/h3>\n
Delcher, A, A. Grove, S. Kasif and J. Pearl, \u201cLogarithmic Time Queries and Updates in Probabilistic Networks\u201d<\/a>, Journal of Artificial Intelligence Research, Vol. 4., pp. 37\u201359, 1996<\/span><\/h3>\n
Chapter 6. Computational Genomics (Human Genome Project, Before and After)<\/h3>\n
(With Steven Salzberg, Art Delcher, Owen White, Herve Tettelin, many amazing TIGR biologists, Human Genome Project Consortium and many pioneers in computational genomics, Rich Roberts, Charles Cantor, Yu Zheng, Megon Walker, Chunmin Ding,\u00a0 Charles DeLisi, Zhiping Weng and many, many more.)<\/h4>\n
Selected Results, Ideas, Frameworks:\u00a0<\/span><\/h4>\n
GLIMMER (one of the most widely used gene finders in bacterial genomes)<\/a><\/h4>\n
MUMMER (widely used comparative genome system, first open access whole genome comparison system for bacteria).<\/a><\/h4>\n
PUBLIC HUMAN GENOME PROJECT (HGP) – with an amazing group of collaborators that include many computational biology pioneers<\/h3>\n
BAYES NETS FOR ANALYSIS OF SPLICING (with Art Delcher)<\/a><\/h4>\n
Bayes Networks for Genomic Data Integration and Genome Annotation<\/a><\/h3>\n
REMOTE HOMOLOGY PREDICTION USING SIMPLE KERNELS FOR Human Genome Annotation<\/a> (with Pedro Moreno and others at HP Labs, summer 2000)<\/h4>\n
MULTIPLEX PCR (for Prenatal Diagnostics of Down Syndrome and Liquid Biopsy).<\/a><\/h3>\n
(With Charles Cantor, John Rachlin, Noga Alon and others.)<\/h3>\n
Early Network and Genomic Data Integration for Genome Annotation and Gene Function Prediction<\/a><\/h3>\n
(with Rich Roberts and Yu Zheng)<\/h3>\n
COMPARATIVE ANALYSIS (many papers in human, mouse, model organisms, bacterial genomes).<\/h3>\n
and more …<\/h4>\n
Lesson 10: <\/span><\/h2>\n
We were very lucky to be part of this transformative period but there is a quote that captures this better than most.<\/span><\/h3>\n
“Fortune favors the prepared mind’, Louis Pasteur<\/span><\/h3>\n
Chapter 7. Computational Genomic Network Systems Biology <\/strong><\/h3>\n
(with Rich Roberts, Yu Zheng, Stan Letovsky and many others)<\/h3>\n
1. Network Based Function Prediction (2002-)<\/span><\/h3>\n
Partners: Stan Letovsky, T.M. Murali, Naoki Naryai, Charles Cantor, Rich Roberts,\u00a0 and many others.<\/h4>\n
We helped conceive and popularize systematic whole genome network based function prediction.\u00a0 Our work was derived from frameworks previously used in AI, Computer Vision and CS that evolved from Markov Random Fields,\u00a0 Neural Nets (Hopfield Networks), Graph Cuts, Graph Diffusion and Label Propagation in Networks. Today, network based function prediction is a widely used framework for the field.<\/h3>\n
2. Network Based Disease Biomarkers for Diabetes and Metabolic Disease<\/h3>\n
Partners: Zak Kohane, Ron Kahn, Manway Liu, Terrence Wu, Tianxi Cai and others as part of the I2B2 NIH National Center at Harvard Partners)<\/h4>\n
3. Multiplex Networks,\u00a0 Biological Context Networks, Multi-node graphs (2004-)<\/span><\/h3>\n
(With Noga Alon, Vera Asodi, Charles Cantor, John Rachlin and others)<\/h4>\n
4. Regulatory Networks\u00a0 Discovery and Experimental Validation (2001-)<\/h3>\n
(With Geoff Cooper, John Tullai, Michael Schaffer, Jim Collins, Tim Gardner, J. Faith, Boris Hayette, Esther Rheinbay, Mario Suva, Brad Bernstein and many more.)<\/h4>\n
Chapter 8. Rich Roberts and the Computational Bridges to Experiments (COMBREX) Project<\/span><\/h3>\n
Chapter 9.\u00a0 I2B2 (Informatics to Bedside) at Harvard Partners<\/h3>\n
(with Zak Kohane, Tianxi Cai,\u00a0 ME Patti, Ron Kahn, Terrence Wu and many others).<\/h3>\n
Chapter 10. Joslin Diabetes NIH Center and Regional Systems Biology Core (Diabetes Type 2, Aging)<\/h3>\n
(With Michael Molla, Jonathan Dreyfuss, Ron Kahn, ME Patti, Allison Goldfine, George King and others)<\/h3>\n
Lesson 11:<\/span><\/h3>\n
Based on my experience working with Joslin\u00a0 and inspired by the quote “Make things as simple as possible\u00a0 but not any simpler”, it is tempting to develop a theory of wellness which is minimal, supported by evolution and enables direct control.\u00a0<\/span><\/span><\/h3>\n
Chapter 11. AI2BIO<\/span><\/h2>\n
Lesson : <\/span><\/h2>\n
After working for almost 30 years at the intersection of AI and Biology I believe that the integration of AI and Biology has unlimited potential. This goes way beyond the old arguments about biology inspired AI (e.g., evolutionary algorithms, neural computation, etc).\u00a0 I am alluding to the transformative methodologies developed in AI that can be used to understand nature.\u00a0 In fact, in a recent talk at MIT I have argued “Should Machines Understand Nature to Pass the Turing Test? Co-evolving AI and Systems\/Synthetic Biology”<\/a><\/span><\/h3>\n
AI and Biology (with Rich Roberts)<\/h3>\n
AI and Ethics (review)<\/a><\/h3>\n
Tracing Predictions to Experimental Ground Truth (With Rich Roberts)<\/a><\/h3>\n
Chapter 12: ?<\/h3>\n
<\/h3>\n