Jesse Thomason

Postdoc @ University of Washington, 2018–...
PhD @ University of Texas at Austin, 2013–2018
thomason DOT jesse AT gmail

I am a postdoctoral researcher working with Luke Zettlemoyer at the University of Washington. My research focuses on language grounding and natural language processing applications for robotics (RoboNLP), including how dialog with humans can facilitate both robot task execution and learning.

News

June 2019
I am co-chairing the NAACL 2019 Combined Workshop on Spatial Language Understanding (SpLU) and Grounded Communication for Robotics (RoboNLP) in Minneapolis, MN on June 6. We will identify shared and unique challenges, problems, and future directions across spatial language, grounding, and robotics.
July 2018
I am co-organizing the SIGdial 2018 Special Session on Physically Situated Dialogue in Melbourne, Australia on July 12th. We received a great turnout of exciting submissions to the special session track, and look forward to hosting a language and robotics-focused event within the dialog community.
June 2018
I am co-organizing the Robotics: Science and Systems 2018 Workshop on Models and Representations for Natural Human-Robot Communication in Pittsburgh, PA on June 29th and 30th. We are planning a two-day workshop with eight invited speakers, together with contributed talks and posters.
June 2018
I am starting as a postdoc with Luke Zettlemoyer at the University of Washington. I am excited to work on grounded semantic understanding tasks, and for the opportunity to collaborate with many of the great researchers at UW.
April 2018
I defended my dissertation on Continually Improving Grounded Natural Language Understanding through Human-Robot Dialog, wrapping up nearly 5 years in Austin working with Ray Mooney and collaborating with Peter Stone. I am grateful to them and to my frequent collaborators, Jivko Sinapov, Shiqi Zhang, and Aishwarya Padmakumar!

Preprints

2019
Interpreting Black Box Models with Statistical Guarantees
Collin Burns, Jesse Thomason, and Wesley Tansey.
arXiv, 2019.
paper
@inproceedings{burns:arxiv19,
  title={Interpreting Black Box Models with Statistical Guarantees},
  author={Collin Burns and Jesse Thomason and Wesley Tansey},
  booktitle={arXiv},
  year={2019}
}

Conference and Journal Papers

2019
Improving Robot Success Detection using Static Object Data
Rosario Scalise, Jesse Thomason, Yonatan Bisk, and Siddhartha Srinivasa.
International Conference on Intelligent Robots and Systems (IROS), 2019.

Also presented at the Combined Workshop on Spatial Language Understanding & Grounded Communication for Robotics (SpLU-RoboNLP), 2019.
papervideo
@inproceedings{scalise:iros19,
  title={Improving Robot Success Detection using Static Object Data},
  author={Rosario Scalise and Jesse Thomason and Yonatan Bisk and Siddhartha Srinivasa},
  booktitle={International Conference on Intelligent Robots and Systems (IROS)},
  year={2019}
}
| SpLU-RoboNLP poster
Augmenting Knowledge through Statistical, Goal-oriented Human-Robot Dialog
Saeid Amiri, Sujay Bajracharya, Cihangir Goktolga, Jesse Thomason, and Shiqi Zhang.
International Conference on Intelligent Robots and Systems (IROS), 2019.
@inproceedings{amiri:iros19,
  title={Augmenting Knowledge through Statistical, Goal-oriented Human-Robot Dialog},
  author={Saeid Amiri and Sujay Bajracharya and Cihangir Goktolga and Jesse Thomason and Shiqi Zhang},
  booktitle={International Conference on Intelligent Robots and Systems (IROS)},
  year={2019}
}
Shifting the Baseline: Single Modality Performance on Visual Navigation & QA
Jesse Thomason, Daniel Gordon, and Yonatan Bisk.
Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2019.
paperposter
@inproceedings{thomason:naacl19,
  title={Shifting the Baseline: Single Modality Performance on Visual Navigation \& QA},
  author={Jesse Thomason and Daniel Gordon and Yonatan Bisk},
  booktitle={Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)},
  year={2019}
}
Improving Grounded Natural Language Understanding through Human-Robot Dialog
Jesse Thomason, Aishwarya Padmakumar, Jivko Sinapov, Nick Walker, Yuqian Jiang, Harel Yedidsion, Justin Hart, Peter Stone, and Raymond J. Mooney.
International Conference on Robotics and Automation (ICRA), 2019.

Also presented at the SIGDIAL Special Session on Physically Situated Dialogue (RoboDIAL), 2018.
Also presented at the RSS Workshop on Models and Representations for Natural Human-Robot Communication (MRHRC), 2018.
paperdemoposter
@inproceedings{thomason:icra19,
  title={Improving Grounded Natural Language Understanding through Human-Robot Dialog},
  author={Jesse Thomason and Aishwarya Padmakumar and Jivko Sinapov and Nick Walker and Yuqian Jiang and Harel Yedidsion and Justin Hart and Peter Stone and Raymond J. Mooney},
  booktitle={International Conference on Robotics and Automation (ICRA)},
  year={2019}
}
| RoboDIAL paperRoboDIAL videoMRHRC paperMRHRC poster
Prospection: Interpretable Plans From Language By Predicting the Future
Chris Paxton, Yonatan Bisk, Jesse Thomason, Arunkumar Byravan, and Dieter Fox.
International Conference on Robotics and Automation (ICRA), 2019.
paper
@inproceedings{paxton:icra19,
  title={Prospection: Interpretable Plans From Language By Predicting the Future},
  author={Chris Paxton and Yonatan Bisk and Jesse Thomason and Arunkumar Byravan and Dieter Fox},
  booktitle={International Conference on Robotics and Automation (ICRA)},
  year={2019}
}
2018
Interaction and Autonomy in RoboCup@Home and Building-Wide Intelligence
Justin Hart, Harel Yedidsion, Yuqian Jiang, Nick Walker, Rishi Shah, Jesse Thomason, Aishwarya Padmakumar, Rolando Fernandez, Jivko Sinapov, Raymond Mooney, and Peter Stone.
AI-HRI AAAI Fall Symposium Series (AAAI-FSS), 2018.
paper
@inproceedings{hart:aaai-fss18,
  title={Interaction and Autonomy in RoboCup@Home and Building-Wide Intelligence},
  author={Justin Hart and Harel Yedidsion and Yuqian Jiang and Nick Walker and Rishi Shah and Jesse Thomason and Aishwarya Padmakumar and Rolando Fernandez and Jivko Sinapov and Raymond Mooney and Peter Stone},
  booktitle={AI-HRI AAAI Fall Symposium Series (AAAI-FSS)},
  year={2018}
}
Multi-modal Predicate Identification using Dynamically Learned Robot Controllers
Saeid Amiri, Suhua Wei, Shiqi Zhang, Jivko Sinapov, Jesse Thomason, and Peter Stone.
International Joint Conference on Artificial Intelligence (IJCAI), 2018.
paper
@inproceedings{amiri:ijcai18,
  title={Multi-modal Predicate Identification using Dynamically Learned Robot Controllers},
  author={Saeid Amiri and Suhua Wei and Shiqi Zhang and Jivko Sinapov and Jesse Thomason and Peter Stone},
  booktitle={International Joint Conference on Artificial Intelligence (IJCAI)},
  year={2018}
}
Guiding Exploratory Behaviors for Multi-Modal Grounding of Linguistic Descriptions
Jesse Thomason, Jivko Sinapov, Raymond Mooney, and Peter Stone.
AAAI Conference on Artificial Intelligence (AAAI), 2018.

Also presented at the Workshop on Language Grounding for Robotics (RoboNLP), 2017.
papersourceslides
@inproceedings{thomason:aaai18,
  title={Guiding Exploratory Behaviors for Multi-Modal Grounding of Linguistic Descriptions},
  author={Jesse Thomason and Jivko Sinapov and Raymond Mooney and Peter Stone},
  booktitle={AAAI Conference on Artificial Intelligence (AAAI)},
  year={2018}
}
| RoboNLP paperRoboNLP poster
Maximum-Variance Total Variation Denoising for Interpretable Spatial Smoothing
Wesley Tansey, Jesse Thomason, and James G Scott.
AAAI Conference on Artificial Intelligence (AAAI), 2018.

Also presented at the ICML Workshop on Human Interpretability in Machine Learning (ICML-WHI), 2017.
paperposter
@inproceedings{tansey:aaai18,
  title={Maximum-Variance Total Variation Denoising for Interpretable Spatial Smoothing},
  author={Wesley Tansey and Jesse Thomason and James G Scott},
  booktitle={AAAI Conference on Artificial Intelligence (AAAI)},
  year={2018}
}
| ICML-WHI paperICML-WHI poster
2017
Opportunistic Active Learning for Grounding Natural Language Descriptions
Jesse Thomason, Aishwarya Padmakumar, Jivko Sinapov, Justin Hart, Peter Stone, and Raymond J. Mooney.
Conference on Robot Learning (CoRL), 2017.
paperdemosourceposterslides
@inproceedings{thomason:corl17,
  title={Opportunistic Active Learning for Grounding Natural Language Descriptions},
  author={Jesse Thomason and Aishwarya Padmakumar and Jivko Sinapov and Justin Hart and Peter Stone and Raymond J. Mooney},
  booktitle={Conference on Robot Learning (CoRL)},
  year={2017}
}
Improving Black-box Speech Recognition using Semantic Parsing
Rodolfo Corona, Jesse Thomason, and Raymond J. Mooney.
International Joint Conference on Natural Language Processing (IJCNLP), 2017.
paperposter
@inproceedings{corona:ijcnlp17,
  title={Improving Black-box Speech Recognition using Semantic Parsing},
  author={Rodolfo Corona and Jesse Thomason and Raymond J. Mooney},
  booktitle={International Joint Conference on Natural Language Processing (IJCNLP)},
  year={2017}
}
Multi-Modal Word Synset Induction
Jesse Thomason and Raymond J. Mooney.
International Joint Conference on Artificial Intelligence (IJCAI), 2017.
paperposterslides
@inproceedings{thomason:ijcai17,
  title={Multi-Modal Word Synset Induction},
  author={Jesse Thomason and Raymond J. Mooney},
  booktitle={International Joint Conference on Artificial Intelligence (IJCAI)},
  year={2017}
}
Integrated Learning of Dialog Strategies and Semantic Parsing
Aishwarya Padmakumar, Jesse Thomason, and Raymond J. Mooney.
European Chapter of the Association for Computational Linguistics (EACL), 2017.
paper
@inproceedings{padmakumar:eacl17,
  title={Integrated Learning of Dialog Strategies and Semantic Parsing},
  author={Aishwarya Padmakumar and Jesse Thomason and Raymond J. Mooney},
  booktitle={European Chapter of the Association for Computational Linguistics (EACL)},
  year={2017}
}
BWIBots: A platform for bridging the gap between AI and human--robot interaction research
Piyush Khandelwal, Shiqi Zhang, Jivko Sinapov, Matteo Leonetti, Jesse Thomason, Fangkai Yang, Ilaria Gori, Maxwell Svetlik, Priyanka Khante, Vladimir Lifschitz, J. K. Aggarwal, Raymond Mooney, and Peter Stone.
Sage: The International Journal of Robotics Research (IJRR), 2017.
paper
@article{khandelwal:ijrr17,
  title={BWIBots: A platform for bridging the gap between AI and human--robot interaction research},
  author={Piyush Khandelwal and Shiqi Zhang and Jivko Sinapov and Matteo Leonetti and Jesse Thomason and Fangkai Yang and Ilaria Gori and Maxwell Svetlik and Priyanka Khante and Vladimir Lifschitz and J. K. Aggarwal and Raymond Mooney and Peter Stone},
  journal={The International Journal of Robotics Research (IJRR)},
  publisher={Sage},
  year={2017}
}
2016
Learning Multi-Modal Grounded Linguistic Semantics by Playing "I Spy"
Jesse Thomason, Jivko Sinapov, Maxwell Svetlik, Peter Stone, and Raymond Mooney.
International Joint Conference on Artificial Intelligence (IJCAI), 2016.
paperdemosourceposterslides
@inproceedings{thomason:ijcai16,
  title={Learning Multi-Modal Grounded Linguistic Semantics by Playing ``{I} Spy''},
  author={Jesse Thomason and Jivko Sinapov and Maxwell Svetlik and Peter Stone and Raymond Mooney},
  booktitle={International Joint Conference on Artificial Intelligence (IJCAI)},
  year={2016}
}
2015
Learning to Interpret Natural Language Commands through Human-Robot Dialog
Jesse Thomason, Shiqi Zhang, Raymond Mooney, and Peter Stone.
International Joint Conference on Artificial Intelligence (IJCAI), 2015.
paperdemosourceposterslides
@inproceedings{thomason:ijcai15,
  title={Learning to Interpret Natural Language Commands through Human-Robot Dialog},
  author={Jesse Thomason and Shiqi Zhang and Raymond Mooney and Peter Stone},
  booktitle={International Joint Conference on Artificial Intelligence (IJCAI)},
  year={2015}
}
2014
Integrating Language and Vision to Generate Natural Language Descriptions of Videos in the Wild
Jesse Thomason, Subhashini Venugopalan, and Raymond Mooney.
International Conference on Computational Linguistics (COLING), 2014.
paperposter
@inproceedings{thomason:coling14,
  title={Integrating Language and Vision to Generate Natural Language Descriptions of Videos in the Wild},
  author={Jesse Thomason and Subhashini Venugopalan and Raymond Mooney},
  booktitle={International Conference on Computational Linguistics (COLING)},
  year={2014}
}
2013
Prosodic Entrainment and Tutoring Dialogue Success
Jesse Thomason, Huy Nguyen, and Diane Litman.
International Conference on Artificial Intelligence in Education (AIED), 2013.
paperposter
@inproceedings{thomason:aied13,
  title={Prosodic Entrainment and Tutoring Dialogue Success},
  author={Jesse Thomason and Huy Nguyen and Diane Litman},
  booktitle={International Conference on Artificial Intelligence in Education (AIED)},
  year={2013}
}
Differences in User Responses to a Wizard-of-Oz versus Automated System
Jesse Thomason and Diane Litman.
Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2013.
paperslides
@inproceedings{thomason:naacl13,
  title={Differences in User Responses to a Wizard-of-Oz versus Automated System},
  author={Jesse Thomason and Diane Litman},
  booktitle={Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)},
  year={2013}
}
2012
Exploring Multi-dimensional Data on Mobile Devices with Single Hand Motion and Orientation Gestures
Jesse Thomason and Jingtao Wang.
International Conference on Human-computer Interaction with Mobile Devices and Services Companion (MobileHCI), 2012.
paperdemoposter
@inproceedings{thomason:mobilehci12,
  title={Exploring Multi-dimensional Data on Mobile Devices with Single Hand Motion and Orientation Gestures},
  author={Jesse Thomason and Jingtao Wang},
  booktitle={International Conference on Human-computer Interaction with Mobile Devices and Services Companion (MobileHCI)},
  year={2012}
}
Differentiated service strategies for ad-hoc wireless sensor networks in harsh communication environments
Jesse Thomason, Kenji Yoshigoe, R.B. Lenin, James M. Bridges, and Srini Ramaswamy.
Springer: Wireless Networks 18(5), 2012.
paper
@article{thomason:springer:wn12,
  title={Differentiated service strategies for ad-hoc wireless sensor networks in harsh communication environments},
  author={Jesse Thomason and Kenji Yoshigoe and R.B. Lenin and James M. Bridges and Srini Ramaswamy},
  journal={Wireless Networks},
  volume={18},
  issue={5},
  publisher={Springer},
  year={2012},
  doi={10.1007/s11276-012-0418-3}
}

Thesis work

2018
Continually Improving Grounded Natural Language Understanding through Human-Robot Dialog
Jesse Thomason.
Department of Computer Science, The University of Texas at Austin, 2018.
paperslides
@phdthesis{thomason:thesis18,
  title={Continually Improving Grounded Natural Language Understanding through Human-Robot Dialog},
  author={Jesse Thomason},
  booktitle={Doctoral Dissertation},
  school={Department of Computer Science, The University of Texas at Austin},
  year={2018}
}
2016
Continuously Improving Natural Language Understanding for Robotic Systems through Semantic Parsing, Dialog, and Multi-modal Perception
Jesse Thomason.
Doctoral Dissertation Proposal, 2016.
paperslides
@inproceedings{thomason:proposal16,
  title={Continuously Improving Natural Language Understanding for Robotic Systems through Semantic Parsing, Dialog, and Multi-modal Perception},
  author={Jesse Thomason},
  booktitle={Doctoral Dissertation Proposal},
  year={2016}
}

Toybox

FlashCardr
2016
Twitter Poetry
2015
The Infinite Bestiary
2014