Jesse Thomason

Postdoc @ University of Washington, 2018–...
PhD @ University of Texas at Austin, 2013–2018
thomason DOT jesse AT gmail

I am a postdoctoral researcher working with Luke Zettlemoyer at the University of Washington. My research focuses on language grounding and natural language processing applications for robotics (RoboNLP), including how dialog with humans can facilitate both robot task execution and learning.

I am on the academic job market.

News

Organizer
  ACL
First Workshop on Advances in Language and Vision Research (ALVR)websiteJuly 2020
    
Invited Talk
  NeurIPS–ViGIL
@ Visually Grounded Interaction and Language (ViGIL) WorkshopwebsiteDecember 2019
    
Invited Talk
  University of Utah
@ Utah Robotics Center SeminarNovember 2019
    
Invited Talk
  IROS–SPAR
@ Semantic Policy and Action Representations for Autonomous Robots (SPAR) WorkshopwebsiteNovember 2019
    
Invited Talk
  Microsoft Research
Vision-and-Dialog NavigationslidesvideoJuly 2019
    
Co-Chair
  NAACL
Combined Workshop on Spatial Language Understanding (SpLU) and Grounded Communication for Robotics (RoboNLP)websiteJune 2019
    
Co-Organizer
  SIGdial
Special Session on Physically Situated DialoguewebsiteJuly 2018
    
Co-Organizer
  RSS
Workshop on Models and Representations for Natural Human-Robot CommunicationwebsiteJune 2018
    
New Position
  UW
Postdoc with Luke ZettlemoyerJune 2018
    
Dissertation Defense
  UT Austin
Continually Improving Grounded Natural Language Understanding through Human-Robot DialogApril 2018
    

Preprints

2019
Interpreting Black Box Models with Statistical Guarantees
Collin Burns, Jesse Thomason, and Wesley Tansey.
arXiv, 2019.
paper
@inproceedings{burns:arxiv19,
  title={Interpreting Black Box Models with Statistical Guarantees},
  author={Collin Burns and Jesse Thomason and Wesley Tansey},
  booktitle={arXiv},
  year={2019}
}

Conference and Journal Papers

2019 (to appear)
Jointly Improving Parsing and Perception for Natural Language Commands through Human-Robot Dialog
Jesse Thomason, Aishwarya Padmakumar, Jivko Sinapov, Nick Walker, Yuqian Jiang, Harel Yedidsion, Justin Hart, Peter Stone, and Raymond J. Mooney.
The Journal of Artificial Intelligence Research (JAIR), 2019 (to appear).
@article{thomason:jair19,
  title={Jointly Improving Parsing and Perception for Natural Language Commands through Human-Robot Dialog},
  author={Jesse Thomason and Aishwarya Padmakumar and Jivko Sinapov and Nick Walker and Yuqian Jiang and Harel Yedidsion and Justin Hart and Peter Stone and Raymond J. Mooney},
  journal={The Journal of Artificial Intelligence Research (JAIR)},
  year={2019 (to appear)}
}
2019
Vision-and-Dialog Navigation
Jesse Thomason, Michael Murray, Maya Cakmak, and Luke Zettlemoyer.
Conference on Robot Learning (CoRL), 2019.
paperwebsitedemosourcepostervideo
@inproceedings{thomason:corl19,
  title={Vision-and-Dialog Navigation},
  author={Jesse Thomason and Michael Murray and Maya Cakmak and Luke Zettlemoyer},
  booktitle={Conference on Robot Learning (CoRL)},
  year={2019}
}
Improving Robot Success Detection using Static Object Data
Rosario Scalise, Jesse Thomason, Yonatan Bisk, and Siddhartha Srinivasa.
Intelligent Robots and Systems (IROS), 2019.

Also presented at the Combined Workshop on Spatial Language Understanding & Grounded Communication for Robotics (SpLU-RoboNLP), 2019.
papervideo
@inproceedings{scalise:iros19,
  title={Improving Robot Success Detection using Static Object Data},
  author={Rosario Scalise and Jesse Thomason and Yonatan Bisk and Siddhartha Srinivasa},
  booktitle={Intelligent Robots and Systems (IROS)},
  year={2019}
}
| SpLU-RoboNLP poster
Augmenting Knowledge through Statistical, Goal-oriented Human-Robot Dialog
Saeid Amiri, Sujay Bajracharya, Cihangir Goktolga, Jesse Thomason, and Shiqi Zhang.
Intelligent Robots and Systems (IROS), 2019.
paperslidesvideo
@inproceedings{amiri:iros19,
  title={Augmenting Knowledge through Statistical, Goal-oriented Human-Robot Dialog},
  author={Saeid Amiri and Sujay Bajracharya and Cihangir Goktolga and Jesse Thomason and Shiqi Zhang},
  booktitle={Intelligent Robots and Systems (IROS)},
  year={2019}
}
Shifting the Baseline: Single Modality Performance on Visual Navigation & QA
Jesse Thomason, Daniel Gordon, and Yonatan Bisk.
North American Chapter of the Association for Computational Linguistics (NAACL), 2019.
paperposter
@inproceedings{thomason:naacl19,
  title={Shifting the Baseline: Single Modality Performance on Visual Navigation \& QA},
  author={Jesse Thomason and Daniel Gordon and Yonatan Bisk},
  booktitle={North American Chapter of the Association for Computational Linguistics (NAACL)},
  year={2019}
}
Improving Grounded Natural Language Understanding through Human-Robot Dialog
Jesse Thomason, Aishwarya Padmakumar, Jivko Sinapov, Nick Walker, Yuqian Jiang, Harel Yedidsion, Justin Hart, Peter Stone, and Raymond J. Mooney.
International Conference on Robotics and Automation (ICRA), 2019.

Also presented at the SIGDIAL Special Session on Physically Situated Dialogue (RoboDIAL), 2018.
Also presented at the RSS Workshop on Models and Representations for Natural Human-Robot Communication (MRHRC), 2018.
paperdemoposter
@inproceedings{thomason:icra19,
  title={Improving Grounded Natural Language Understanding through Human-Robot Dialog},
  author={Jesse Thomason and Aishwarya Padmakumar and Jivko Sinapov and Nick Walker and Yuqian Jiang and Harel Yedidsion and Justin Hart and Peter Stone and Raymond J. Mooney},
  booktitle={International Conference on Robotics and Automation (ICRA)},
  year={2019}
}
| RoboDIAL paperRoboDIAL videoMRHRC paperMRHRC poster
Prospection: Interpretable Plans From Language By Predicting the Future
Chris Paxton, Yonatan Bisk, Jesse Thomason, Arunkumar Byravan, and Dieter Fox.
International Conference on Robotics and Automation (ICRA), 2019.
paper
@inproceedings{paxton:icra19,
  title={Prospection: Interpretable Plans From Language By Predicting the Future},
  author={Chris Paxton and Yonatan Bisk and Jesse Thomason and Arunkumar Byravan and Dieter Fox},
  booktitle={International Conference on Robotics and Automation (ICRA)},
  year={2019}
}
2018
Interaction and Autonomy in RoboCup@Home and Building-Wide Intelligence
Justin Hart, Harel Yedidsion, Yuqian Jiang, Nick Walker, Rishi Shah, Jesse Thomason, Aishwarya Padmakumar, Rolando Fernandez, Jivko Sinapov, Raymond Mooney, and Peter Stone.
AI-HRI AAAI Fall Symposium Series (AAAI-FSS), 2018.
paper
@inproceedings{hart:aaai-fss18,
  title={Interaction and Autonomy in RoboCup@Home and Building-Wide Intelligence},
  author={Justin Hart and Harel Yedidsion and Yuqian Jiang and Nick Walker and Rishi Shah and Jesse Thomason and Aishwarya Padmakumar and Rolando Fernandez and Jivko Sinapov and Raymond Mooney and Peter Stone},
  booktitle={AI-HRI AAAI Fall Symposium Series (AAAI-FSS)},
  year={2018}
}
Multi-modal Predicate Identification using Dynamically Learned Robot Controllers
Saeid Amiri, Suhua Wei, Shiqi Zhang, Jivko Sinapov, Jesse Thomason, and Peter Stone.
International Joint Conference on Artificial Intelligence (IJCAI), 2018.
paper
@inproceedings{amiri:ijcai18,
  title={Multi-modal Predicate Identification using Dynamically Learned Robot Controllers},
  author={Saeid Amiri and Suhua Wei and Shiqi Zhang and Jivko Sinapov and Jesse Thomason and Peter Stone},
  booktitle={International Joint Conference on Artificial Intelligence (IJCAI)},
  year={2018}
}
Guiding Exploratory Behaviors for Multi-Modal Grounding of Linguistic Descriptions
Jesse Thomason, Jivko Sinapov, Raymond Mooney, and Peter Stone.
Conference on Artificial Intelligence (AAAI), 2018.

Also presented at the Workshop on Language Grounding for Robotics (RoboNLP), 2017.
papersourceslides
@inproceedings{thomason:aaai18,
  title={Guiding Exploratory Behaviors for Multi-Modal Grounding of Linguistic Descriptions},
  author={Jesse Thomason and Jivko Sinapov and Raymond Mooney and Peter Stone},
  booktitle={Conference on Artificial Intelligence (AAAI)},
  year={2018}
}
| RoboNLP paperRoboNLP poster
Maximum-Variance Total Variation Denoising for Interpretable Spatial Smoothing
Wesley Tansey, Jesse Thomason, and James G Scott.
Conference on Artificial Intelligence (AAAI), 2018.

Also presented at the ICML Workshop on Human Interpretability in Machine Learning (ICML-WHI), 2017.
paperposter
@inproceedings{tansey:aaai18,
  title={Maximum-Variance Total Variation Denoising for Interpretable Spatial Smoothing},
  author={Wesley Tansey and Jesse Thomason and James G Scott},
  booktitle={Conference on Artificial Intelligence (AAAI)},
  year={2018}
}
| ICML-WHI paperICML-WHI poster
2017
Opportunistic Active Learning for Grounding Natural Language Descriptions
Jesse Thomason, Aishwarya Padmakumar, Jivko Sinapov, Justin Hart, Peter Stone, and Raymond J. Mooney.
Conference on Robot Learning (CoRL), 2017.
paperdemosourceposter
@inproceedings{thomason:corl17,
  title={Opportunistic Active Learning for Grounding Natural Language Descriptions},
  author={Jesse Thomason and Aishwarya Padmakumar and Jivko Sinapov and Justin Hart and Peter Stone and Raymond J. Mooney},
  booktitle={Conference on Robot Learning (CoRL)},
  year={2017}
}
Improving Black-box Speech Recognition using Semantic Parsing
Rodolfo Corona, Jesse Thomason, and Raymond J. Mooney.
International Joint Conference on Natural Language Processing (IJCNLP), 2017.
paperposter
@inproceedings{corona:ijcnlp17,
  title={Improving Black-box Speech Recognition using Semantic Parsing},
  author={Rodolfo Corona and Jesse Thomason and Raymond J. Mooney},
  booktitle={International Joint Conference on Natural Language Processing (IJCNLP)},
  year={2017}
}
Multi-Modal Word Synset Induction
Jesse Thomason and Raymond J. Mooney.
International Joint Conference on Artificial Intelligence (IJCAI), 2017.
paperposterslides
@inproceedings{thomason:ijcai17,
  title={Multi-Modal Word Synset Induction},
  author={Jesse Thomason and Raymond J. Mooney},
  booktitle={International Joint Conference on Artificial Intelligence (IJCAI)},
  year={2017}
}
Integrated Learning of Dialog Strategies and Semantic Parsing
Aishwarya Padmakumar, Jesse Thomason, and Raymond J. Mooney.
European Chapter of the Association for Computational Linguistics (EACL), 2017.
paper
@inproceedings{padmakumar:eacl17,
  title={Integrated Learning of Dialog Strategies and Semantic Parsing},
  author={Aishwarya Padmakumar and Jesse Thomason and Raymond J. Mooney},
  booktitle={European Chapter of the Association for Computational Linguistics (EACL)},
  year={2017}
}
BWIBots: A platform for bridging the gap between AI and human--robot interaction research
Piyush Khandelwal, Shiqi Zhang, Jivko Sinapov, Matteo Leonetti, Jesse Thomason, Fangkai Yang, Ilaria Gori, Maxwell Svetlik, Priyanka Khante, Vladimir Lifschitz, J. K. Aggarwal, Raymond Mooney, and Peter Stone.
The International Journal of Robotics Research (IJRR), 2017.
paper
@article{khandelwal:ijrr17,
  title={BWIBots: A platform for bridging the gap between AI and human--robot interaction research},
  author={Piyush Khandelwal and Shiqi Zhang and Jivko Sinapov and Matteo Leonetti and Jesse Thomason and Fangkai Yang and Ilaria Gori and Maxwell Svetlik and Priyanka Khante and Vladimir Lifschitz and J. K. Aggarwal and Raymond Mooney and Peter Stone},
  journal={The International Journal of Robotics Research (IJRR)},
  publisher={Sage},
  year={2017}
}
2016
Learning Multi-Modal Grounded Linguistic Semantics by Playing "I Spy"
Jesse Thomason, Jivko Sinapov, Maxwell Svetlik, Peter Stone, and Raymond Mooney.
International Joint Conference on Artificial Intelligence (IJCAI), 2016.
paperdemosourceposterslides
@inproceedings{thomason:ijcai16,
  title={Learning Multi-Modal Grounded Linguistic Semantics by Playing ``{I} Spy''},
  author={Jesse Thomason and Jivko Sinapov and Maxwell Svetlik and Peter Stone and Raymond Mooney},
  booktitle={International Joint Conference on Artificial Intelligence (IJCAI)},
  year={2016}
}
2015
Learning to Interpret Natural Language Commands through Human-Robot Dialog
Jesse Thomason, Shiqi Zhang, Raymond Mooney, and Peter Stone.
International Joint Conference on Artificial Intelligence (IJCAI), 2015.
paperdemosourceposterslides
@inproceedings{thomason:ijcai15,
  title={Learning to Interpret Natural Language Commands through Human-Robot Dialog},
  author={Jesse Thomason and Shiqi Zhang and Raymond Mooney and Peter Stone},
  booktitle={International Joint Conference on Artificial Intelligence (IJCAI)},
  year={2015}
}
2014
Integrating Language and Vision to Generate Natural Language Descriptions of Videos in the Wild
Jesse Thomason, Subhashini Venugopalan, Sergio Guadarrama, Kate Saenko, and Raymond Mooney.
Conference on Computational Linguistics (COLING), 2014.
paperposter
@inproceedings{thomason:coling14,
  title={Integrating Language and Vision to Generate Natural Language Descriptions of Videos in the Wild},
  author={Jesse Thomason and Subhashini Venugopalan and Sergio Guadarrama and Kate Saenko and Raymond Mooney},
  booktitle={Conference on Computational Linguistics (COLING)},
  year={2014}
}
2013
Prosodic Entrainment and Tutoring Dialogue Success
Jesse Thomason, Huy Nguyen, and Diane Litman.
Artificial Intelligence in Education (AIED), 2013.
paperposter
@inproceedings{thomason:aied13,
  title={Prosodic Entrainment and Tutoring Dialogue Success},
  author={Jesse Thomason and Huy Nguyen and Diane Litman},
  booktitle={Artificial Intelligence in Education (AIED)},
  year={2013}
}
Differences in User Responses to a Wizard-of-Oz versus Automated System
Jesse Thomason and Diane Litman.
North American Chapter of the Association for Computational Linguistics (NAACL), 2013.
paperslides
@inproceedings{thomason:naacl13,
  title={Differences in User Responses to a Wizard-of-Oz versus Automated System},
  author={Jesse Thomason and Diane Litman},
  booktitle={North American Chapter of the Association for Computational Linguistics (NAACL)},
  year={2013}
}
2012
Exploring Multi-dimensional Data on Mobile Devices with Single Hand Motion and Orientation Gestures
Jesse Thomason and Jingtao Wang.
Human-computer Interaction with Mobile Devices and Services Companion (MobileHCI), 2012.
paperdemoposter
@inproceedings{thomason:mobilehci12,
  title={Exploring Multi-dimensional Data on Mobile Devices with Single Hand Motion and Orientation Gestures},
  author={Jesse Thomason and Jingtao Wang},
  booktitle={Human-computer Interaction with Mobile Devices and Services Companion (MobileHCI)},
  year={2012}
}
Differentiated service strategies for ad-hoc wireless sensor networks in harsh communication environments
Jesse Thomason, Kenji Yoshigoe, R.B. Lenin, James M. Bridges, and Srini Ramaswamy.
Wireless Networks 18(5), 2012.
paper
@article{thomason:springer:wn12,
  title={Differentiated service strategies for ad-hoc wireless sensor networks in harsh communication environments},
  author={Jesse Thomason and Kenji Yoshigoe and R.B. Lenin and James M. Bridges and Srini Ramaswamy},
  journal={Wireless Networks},
  volume={18},
  issue={5},
  publisher={Springer},
  year={2012},
  doi={10.1007/s11276-012-0418-3}
}

Thesis work

2018
Continually Improving Grounded Natural Language Understanding through Human-Robot Dialog
Jesse Thomason.
Department of Computer Science, The University of Texas at Austin, 2018.
paperslides
@phdthesis{thomason:thesis18,
  title={Continually Improving Grounded Natural Language Understanding through Human-Robot Dialog},
  author={Jesse Thomason},
  booktitle={Doctoral Dissertation},
  school={Department of Computer Science, The University of Texas at Austin},
  year={2018}
}
2016
Continuously Improving Natural Language Understanding for Robotic Systems through Semantic Parsing, Dialog, and Multi-modal Perception
Jesse Thomason.
Doctoral Dissertation Proposal, 2016.
paperslides
@inproceedings{thomason:proposal16,
  title={Continuously Improving Natural Language Understanding for Robotic Systems through Semantic Parsing, Dialog, and Multi-modal Perception},
  author={Jesse Thomason},
  booktitle={Doctoral Dissertation Proposal},
  year={2016}
}

Toybox

FlashCardr
2016
Twitter Poetry
2015
The Infinite Bestiary
2014