Tuesday, January 15, 2013

New features for 2013

Happy new year from everyone on the Cognitive Atlas team!  The new year brings an upgrade to the site, with some new features that we are particularly excited about. The biggest new feature is what we call collections, which encompass groups of concepts from the Atlas.  There are currently two types of collections:

Task Batteries are collections of tasks, and are meant to be used to define test batteries such as the NIH Toolbox or the CANTAB.  For example, here is the collections page for the NIH Toolbox Cognition Battery:

The page lists all of the constituent tasks that make up the Cognition Battery, and links to the separate task pages for each of those tasks.  It also allows specification of "derived indicators," which are composite measures that are derived from performance across multiple tasks.  Using this feature, we plan to represent all of the neuropsychological test batteries commonly used today.

Theories are defined as sets of assertions about concepts in the atlas.  For example, here is a (sketchy) representation of Baddeley's theory of working memory:

This is clearly just a start at defining this theory, but we believe that this feature can nonetheless provide a very useful formal summary of many of the theories in the domain of psychology.  Registered users should go ahead and add their favorite theory today!

Task implementations.  We have added a new feature to the task pages which support links to openly available implementations of tasks.  For example, here is the stop signal task page with a couple of implementations:

As always, your thoughts and suggestions are welcome and greatly appreciated!

Saturday, September 24, 2011

A meeting of the minds on mental ontologies

We had an incredibly useful meeting this week at UT which brought together several of the groups that are currently working on different ontologies related to mental function:

A major goal of the meeting was for us all to get familiar with Janna's work on the Emotion Ontology, since it's the newest kid on the block. In addition, since Janna is a card-carrying ontologist, I particularly wanted to get comments from her about the structure of our new Cognitive Atlas OWL ontology.  

The major challenge for all of us is how we interoperate while still addressing the needs of our individual projects.  Three of the ontologies (Cognitive Atlas, CogPO, and NEMO) have parallel representations of cognitive tasks, which have different uses in each case and different semantics; in the end we would like to have a single ontology that represents all of these, but until then our approach will be to identify (automatically, we hope) the matching tasks between our databases and expose a translation table that will map the task identifiers between the three databases.  We also worked on aligning the annotation features that each of the databases uses in our OWL representations, to make integration more straightforward in the future.

Much of our discussion centered around the kinds of philosophical issues that are at the heart of why we created the Cognitive Atlas, namely the question of whether and how ontology creation can be community based (i.e., the "bazaar" approach, after Eric Raymond) versus being driven by a single small group of individuals (the "cathedral" approach).  The Cognitive Atlas follows the Wikipedia-like approach of allowing (indeed, encouraging) and capturing disagreement, whereas the other projects are all more heavily driven by small groups of individuals and attempt to capture a single scheme with great clarity.  Both approaches have their advantages, and one of our major challenges is to figure out how to bridge the two.  In the near future we will be attempting to import the Emotion Ontology into the Cognitive Atlas knowledge base, which will provide us with our first test regarding interoperability between these knowledge bases.

We spent a bit of time discussing the current structure of the Cognitive Atlas from the standpoint of the hardcore ontologists.  One of the issues that was noted was the relatively flat structure of the knowledge base at present; that is, there are lots of concepts but relatively few relations between them.  That is evolving, but it highlights the need for more action on the relations side.  Another issue that is more difficult is the need for less ambiguous definitions. Right now, the definitions in Cognitive Atlas are often quite ambiguous, such that it would not be possible for a person to uniquely identify which concept was the correct one in a particular situation.  The major problem here is that the definitions are being written by domain experts, not by ontologists, but we are hopeful that as we continue to interact with the ontology community, we can develop better definitions and provide better guidance to our users as to best practices for unambiguous definitions.

Overall this was a great meeting and I'm looking forward to the next one! It's really exciting to see so many groups interested in cognitive ontologies, and to see us all working together so closely.

Wednesday, August 17, 2011

Cognitive Atlas Paper accepted at Frontiers in Neuroinformatics

Our paper describing the Cognitive Atlas project in detail has just been accepted at Frontiers in Neuroinformatics, and the provisional PDF is accessible from here.  The paper lays out the project in much more detail than we have elsewhere, and I would recommend that anyone planning to work seriously on the project first read this paper.

Friday, August 5, 2011

New Release of Cognitive Atlas

Yesterday we pushed the latest version of the Cognitive Atlas to the production site, after many months of development.  I'd like to give a quick tour of some of the new features.

Overall organization.  In the new version we have reduced the prominence of assertions as a separate feature. Instead, assertions are simply integrated with tasks and concepts via the relation browsers. 

Dashboard.  When you log into your account, you are now taken to a dashboard that shows the latest activity on the site, recent comments, and most viewed concepts, tasks, and assertions.

Concept Browser.  There are now two views of each concept under separate tabs on the concept page.  The atlas view provides a graphical view of the relations between a concept and other concepts, while the list view provides a simpler list.  This new system should scale much better to larger numbers of relations.  We have also separated the display of measurement relations into a different panel from other ontological relations.

Task Browser.  We have a new, much more compact view of task conditions, concepts, and indicators.  We also have new tabbed panes for measurement relations and task phylogeny relations.  Here again the goal was primarily to allow the display to scale to cases where there is a large amount of information.

Neurosynth integration. We have included a new tab in the visualization panel (next to the PubMed tab) for terms that are included in the Neurosynth database, which links directly to those pages.  In the future we plan to have direct integration of the neurosynth browser into this panel.

Concept forking. We now have a button on each page that allows one to "fork" a concept, which is the last resort when a term has more than one different sense.  Use this with care!

Curation features.  Each concept and task page now has a button labeled "Flag this item for review by the curator" which will notify us that the page needs to be examined by a curator.  As we being to exert more effort on the curation side, this will be an important behind-the-scenes feature.

Please take a look at the new site and give us feedback about the new features or any bugs that you find.  

Thursday, June 30, 2011

OWL-izing the Cognitive Atlas

Recently I have spent some time turning the Cognitive Atlas knowledge base into a full-blown ontology.  This effort came about as I was working on responding to reviews from a paper describing the project (submitted to Frontiers in Neuroinformatics).  We made the choice at the outset of the project to store it in a custom relational database, rather than using an ontology language as the basic storage format, because we wanted to retain flexibility to store things that are not necessarily ontology-friendly.  We always said that we would come up with a way to later translate the knowledge into a formal ontology, but I began to get a bit itchy about our ability to actually do this.  So, over the last few weeks I have created a pipeline for turning the database into an OWL ontology.  I have made the code and ontology available at https://github.com/poldrack/cogat in case anyone is interested in having a look.  I am a complete newbie when it comes to ontologies, so any comments on the structure of the ontology are very welcome. (We would also welcome comments on the content of the ontology, but if you don't like the content then you really should just go into the Cognitive Atlas site and change it yourself!)

I started with RDF dumps of the concepts and tasks, which are available from the SPARQL endpoint page.  Assertions about relations in the database are not currently available from the endpoint, so I got a custom dump of those as well.  I then wrote a python script to finesse all of this into an OWL ontology. I have to say that this is a great way to learn about the structure of OWL!  I used Protégé to help validate my OWL and examine the contents, which was really useful.  

Part of the work in setting up this ontology was deciding how to align it with other ontologies, particularly with CogPO. I had a long meeting at OHBM2011 with Jessica Turner and Angie Laird, during which we hashed out a set of classes and relations that will serve to structure the task representation in both of our ontologies.  For our representation of tasks, we are inheriting the BehavioralExperimentalParadigm class from CogPO. 

I have also registered the Cognitive Atlas with the NCBO BioPortal site, which I hope will give it increased visibility.  In the long run, I plan to create a pipeline by which the ontology will the automatically generated and updated directly from the database, but for now will do regular manual updates.

Saturday, April 9, 2011

Some updates on the Cognitive Atlas Project

It's been a while since I posted anything so I thought I would catch everyone up on what's been going on.  The project ended up having a number of very quiet months last year while we transferred the grant from UCLA to UT Austin.  Now that the grant is in place at UT, we have been cranking forward on the next release of the Cognitive Atlas, which we hope to release some time later this year.  Some new features will include:

  • a personal "dashboard" that will provide easy access to relevant infomation
  • improved interface for displaying relations, which will work better when there are large numbers of relations
  • an improved interface to PubMed
  • relations based on literature mining
  • many more improvements behind the scenes
We have also welcomed Jack Van Horn from UCLA to the Cognitive Atlas team; Jack will be working with the rest of the team to develop new ways to seed the database and visualize the results.  

I have also just recently made a thorough curation run through the database and will soon be cleaning out the database.  Discussions with a number of people (particularly Gwen Friskoff from Georgia State) highlighted that there were some terms in the database that were not really mental constructs.  For example, there were a number of linguistic concepts (such as "vowel" and "agglutinative language") which refer to things in the world rather than to mental constructs.  Because the concepts list in the Cognitive Atlas is meant to include only mental constructs, these are not appropriate for the knowledge base.  Instead, a concept like "vowel processing" would be appropriate because it would refer to a mental construct rather than to a thing in the world.  

Another common confusion is between tasks and concepts. For example, "trace conditioning" was previously listed as a concept in the database.  However, trace conditioning is an empirical phenomenon that is evoked using particular learning paradigm; it is, of course, supported by particular learning processes, and it is those processes that count as mental concepts, rather than the experimental paradigm, which should instead be listed as a task.  

Finally, we are in the process of making a thorough sweep through the task database, integrating with a number of other task lexica, and cleaning it up so that entries are more consistent.  

We look forward to your continued contributions and always welcome feedback about the project.

Thursday, July 15, 2010

NIMH Research Domain Criteria Meeting

I spent the last two days in Bethesda at the first workshop of the NIMH RDoC project, which was one of the most interesting scientific meetings I've been to in quite a while.  This project is meant to address two major problems in psychiatry.  First, there is widespread dissatisfaction with the current categorical approaches to psychiatric diagnoses, and a growing realization that better understanding of these disorders requires a dimensional (rather than categorical) perspective (e.g., Kraemer, 2007).  Second, there is  an increasing realization of the need to ground psychiatric diagnoses in neuroscience (Miller, 2010). The goal of the RDoC project is to help determine what the best candidates are for neuroscientifically validated dimensions that are most likely to be relevant to mental disorders.

This workshop was the first of several, each of which will focus on a particular construct or domain.  This first meeting was focused on the concept of "working memory."  The group was quite broad, from researchers who study working memory in mental disorders to neurobiologists who study the molecular basis of sustained activity in rodents.  The first charge of the group was to come up with a definition of working memory.  Given how prevalent this concept is in the literature (more than 11,000 hits in PubMed), it might seem that a definition would be obvious, but it took the better part of an hour for each of the two  working groups to come up with a definition that most of us could agree on.  Remarkably, the definitions generated independently by each group were very similar!  A summary of the consensus definition (generated through additional discussion of the whole group) has been entered as the new definition of the "working memory" concept in the Cognitive Atlas and can be found here

Much of the remainder of the workshop focused on characterizing the tasks, neural systems, molecules, and genes that are relevant to working memory.  Although each group came up with lists of tasks that were thought to best capture the construct of working memory, there was some trepidation about publishing this list, lest it become a blessing for the uncritical use of these tasks as "working memory tasks".  Rather, the group focused more on discussing what task features are most important for ensuring construct validity of tasks meant to measure working memory.  These will be published soon as part of the output from the workshop.

Another interesting point of discussion centered around how to relate the different levels of analysis, from molecules to neuropsychological tests.  The challenge comes in thinking about how construct mechanistically realistic links between molecular constructs (such as allelic variation in the COMT gene) and behavior (such as performance on the N-back task).  Drawing from the approach that has been developed within the UCLA Consortium for Neuropsychiatric Phenomics, Bob Bilder and I argued that this should be thought of in terms of links between successive levels of structure. For example, we can establish links between cognitive processes and neural systems, between neural systems and cellular systems, and so on down to genes and environmental influences. These different levels can then be traversed to provide a sophisticated mechanistic understanding of how we can get from genes to cognitive function.

Finally, there was a good bit of interesting discussion about the need for measures of working memory that are better validated, both conceptually and psychometrically. There was a clear undercurrent of dissatisfaction with the degree to which the imaging literature has relied on tasks that are not well validated in either of these senses (the N-back task being the poster child for this).  There seems to be great interest in developing a battery of validated tests, and maybe even greater interest in the potential of using these for online testing of large groups across studies.  For example, this seems to be the only realistic way to get sample sizes large enough to make well-powered genome-wide association analyses feasible for cognitive phenotypes.

The RDoC meeting was particularly exciting because of how well it dovetails with the Cognitive Atlas Project.  Getting researchers involved in clarifying their concepts and specifying how they relate to tasks and other levels of analysis is exactly what this project was designed to do.  I have already entered a good bit of the knowledge that was generated in this first RDoC workshop into the system, and I hope that we can make the Cognitive Atlas a repository for all of the knowledge that is generated in the subsequent RDoC workshops as well.  It was heartening to see that the framework that we have developed for representing this kind of knowledge is so well aligned to the kinds of knowledge building that NIMH clearly sees as essential for progress.