CogSketch Tutorial

organized by
Kenneth D. Forbus, Maria Chang, and Andrew Lovett
Qualitative Reasoning Group, Northwestern University

People sketch to work out ideas and to communicate with each other. This makes it a natural window into exploring spatial cognition, and for creating software that works more naturally with people. Sketch understanding software can be used to help model aspects of human cognition, to gather data from human participants in experiments, and in developing new kinds of intelligent software. CogSketch, a publicly available sketch understanding system, is being developed to support cognitive science research and to serve as a platform for sketch-based educational software. CogSketch incorporates visual processing of digital ink, qualitative spatial representations, analogical matching over integrated spatial and conceptual representations, and a large open-source knowledge base. It has already been used by psychologists and learning scientists for gathering data in laboratory experiments, by cognitive scientists to simulate human visual reasoning,  and in classroom experiments in geoscience and engineering.  This half-day tutorial is intended for cognitive science researchers who want to use sketching in their research.  Attendees will learn the basics of CogSketch and how it might be used to facilitate their research. This includes a summary of its representations and processing, and the support it provides for laboratory experiments, computational experiments, and building educational software. You can find out more about CogSketch, and download it, at

We encourage participants to bring notebook computers with CogSketch installed, so they can work through examples, but this will not be a requirement. 

The topics to be covered include:

  • Basics of open-domain sketch understanding. Digital ink, conceptual labeling. 
  • Visual processing in CogSketch. Ink and Voronoi processing. Qualitative spatial representations and grouping operations. Decomposition and matching of shapes. 
  • Analogical processing in CogSketch. Basics of structure-mapping theory, structure-mapping engine (SME).  Spatial considerations in mapping. 
  • Support for laboratory experiments. Examples of experiments already conducted using CogSketch. Copy-and-paste of stimuli from PowerPoint.  Accessing timing information.  Gathering sketch data.  Automatic data interpretation and scoring.
  • Support for education research.  The worksheet model and authoring environment.  “Phone-home” data gathering and analysis facilities.
  • Support for computational experiments. Exporting knowledge and ink data. Overview of CogSketch API for using its spatial reasoning facilities from other software. Extending the knowledge base.

The exact depth of the last three topics will depend on the mix of interests of the attendees.  Please contact us via cogsketch-team at CS dot  northwestern dot edu, with the subject line SC 2014, to make suggestions. We look forward to seeing you there.