image

Andrew Rosenberg
andrew_at_cs.qc.cuny.edu
PI: Speech Lab @ Queens College
Computer Science Department
New Science Building A332
Queens College / CUNY
CUNY Graduate Center Rm 4420
718-997-3562

News:

  • Most deadlines have passed. Time to get back to work.

I am an Assistant Professor of Computer Science at Queens College (CUNY), and a member of the Doctoral Faculty of the Computer Science and Linguistics programs at the CUNY Graduate Center. I completed my Ph.D. at Columbia University in 2009. I lead the Speech Lab @ Queens College. I am a NSF CAREER Award winner.

My research concerns Natural Language Processing, Spoken Language Processing, Prosody/Intonation and Machine Learning.

I am the main author and maintainer of AuToBI - an open source java toolkit for prosodic analysis in English.

Along with Julia Hirschberg and Stefanie Shattuck-Hufnagel, I am developing Reciprosody, a repository of prosodically annotated speech.

I collaborate part time at the IBM TJ Watson Research Lab, where I helped improve the speech synthesis quality for Watson, the Jeopardy! playing system.

At Interspeech 2011 in Florence, I gave a tutorial called "More than Words Can Say: Prosodic Analysis Techniques and Applications". The slides are available here.

I won the CUNY Junior Faculty Research Award in Science and Engineering sponsored by the Sloan Foundation in 2014. My research statement can be found here.

Current cv

Most recent blog post:

AuToBI

Based on my dissertation, "Automatic Detection and Classification of Prosodic Events", I developed AuToBI, a system to generate hypothesized ToBI labels.
Visit the AuToBI webpage for more details.

Detecting Deception in Speech

People indicate that they are lying in different ways. While one speaker might hesitate a lot while being deceptive, and very little when being truthful, another speaker might demonstrate the opposite behavior. We are investigating the lexical and acoustic correlates to deceptive behavior. In this work, we are paying particular attention to two qualities: cross-cultural differences in deceptive behavior, and 2) the influence of a speaker's personality on their deceptive strategy.

Collaborators: Columbia University, Barnard College

Using Prosody to Improve Keyword Search in Low Resource Languages

The intonation of a spoken sentence can inform the prediction of the words that are being said. For example, questions contain different words than declarative statements. We are developing techniques to use intonational (or prosodic) information to improve keyword search or (spoken term detection) in low resource languages. Collaborators: IBM, Columbia University, New York University, RWTH Aachen, University of Cambridge, University of Southern California

Analysis of Political Speech

Political campaigns are centered around political speeches and debates. Even with big budget television ads, and netroots organizing, political rhetoric remains the most potent device for a politician to drive a message. We are examining the rhetorical qualities of political speech and debate performances to determine what makes a politician attractive to a population.

User Modeling based on Typing Behavior

People have different typing styles. Some people plan a sentence in their head and then type it with no pausing or revision. Others type in shorter spurts with many corrections. We are developing techniques that explore the different physical, lexical and cognitive correlates to pausing and revision in typing behavior. These correlates will then be used to distinguish typists.

Collaborators: New York Institute of Technology, Louisiana Tech

Analysis of Non-native Intonation

Intonation is typically the last aspect of spoken language acquired by non-native speakers. In this work, we examine the influence of Mandarin Chinese speakers' native language on their perception and production of intonation in American English.

V-Measure: Cluster Evaluation Measure

Evaluating the success of clustering algorithms is not a trivial task. In 2007, I proposed a conditional entropy-based measure called V-measure at EMNLP. This measure is described in V-Measure: A conditional entropy-based external cluster evaluation measure. The name comes from its similarity in calculation to F-measure, and its goal of evaluating cluster validity.

V-measure is the evaluation measure for the unsupervised evaluation of the SemEval 2010 Word Sense Induction and Disambiguation Task

Distributed here is a java package that implements V-Measure as well as a number of other cluster evaluation measures. Feel free to email with bug reports and feature requests
ClusterEvaluator.jar
ClusterEvaluator source code
Javadoc for ClusterEvaluator.

Spring 2014
CUNY Graduate Center - Methods in Computational Linguistics II

Fall 2013
CUNY Graduate Center - Natural Language Processing, Machine Learning and The Web

Fall 2012
Queens College - Research Practicum

Spring 2012
Queens College - Machine Learning
CUNY Graduate Center - Methods in Computational Linguistics II

Fall 2011
Queens College - Algorithms I
Queens College - Spoken Language Processing

Spring 2011
Queens College - Algorithms I
CUNY Graduate Center - Machine Learning

Fall 2010
Queens College - Algorithms I
CUNY Graduate Center - Methods in Computational Linguistics I

Spring 2010
Queens College - Machine Learning
CUNY Graduate Center - Machine Learning

Fall 2009
Queens College - Algorithms I

2014

  • Denys Katerenchuk, David Guy Brizan and Andrew Rosenberg. Was that your mother on the phone?: Classifying Interpersonal Relationships between Dialog Participants with Lexical and Acoustic Properties. Interspeech 2014, Singapore [bib] @inproceedings{Katerenchuk:2014:mother,
      Author = {Denys Katerenchuk, David~Guy Brizan, Andrew Rosenberg},
      Booktitle = {Interspeech},
      Title = {Was that your mother on the phone?: Classifying Interpersonal Relationships between Dialog Participants with Lexical and Acoustic Properties},
      Year = {2014}}
  • Denys Katerenchuk and Andrew Rosenberg. Improving Named Entity Recognition with Prosodic Features. Interspeech 2014, Singapore [bib] @inproceedings{Katerenchuk:2014:strategies,
      Author = {Denys Katerenchuk, Andrew Rosenberg},
      Booktitle = {Interspeech},
      Title = {Improving Named Entity Recognition with Prosodic Features},
      Year = {2014}}
  • Min Ma, Justin Richards, Victor Soto, Julia Hirschberg and Andrew Rosenberg. Strategies for Rescoring Keyword Search Results Using Word-Burst and Acoustic Features. Interspeech 2014, Singapore [bib] @inproceedings{ma:2014:strategies,
      Author = {Min Ma, Justin Richards, Victor Soto, Julia Hirschberg, Andrew Rosenberg},
      Booktitle = {Interspeech},
      Title = {Strategies for Rescoring Keyword Search Results Using Word-Burst and Acoustic Features},
      Year = {2014}}
  • Raul Fernandez, Jia Cui, Andrew Rosenberg, Bhuvana Ramabhadran and Xiaodong Cui. Exploiting Vocal-Source Features to Improve ASR Accuracy for Low-Resource Languages. Interspeech 2014, Singapore [bib] @inproceedings{fernandez:2014:exploiting,
      Author = {Raul Fernandez, Jia Cui, Andrew Rosenberg, Bhuvana Ramabhadran, Xiaodong Cui},
      Booktitle = {Interspeech},
      Title = {Exploiting Vocal-Source Features to Improve ASR Accuracy for Low-Resource Languages},
      Year = {2014}}
  • Victor Soto, Lidia Mangu, Andrew Rosenberg and Julia Hirschberg. A Comparison of Multiple Methods for Rescoring Keyword Search Lists for Low Resource Languages. Interspeech 2014, Singapore [bib] @inproceedings{soto:2014:comparison,
      Author = {Victor Soto, Lidia Mangu, Andrew Rosenberg, Julia Hirschberg},
      Booktitle = {Interspeech},
      Title = {A Comparison of Multiple Methods for Rescoring Keyword Search Lists for Low Resource Languages},
      Year = {2014}}
  • Xiaodong Cui, Brian Kingsbury, Jia Cui, Bhuvana Ramabhadran, Andrew Rosenberg, Mohammad Sadegh Rasooli, Owen Rambow, Nizar Habash and Vaibhava Goel. Improving Deep Neural Network Acoustic Modeling For Audio Corpus Indexing Under The IARPA Babel Program. Interspeech 2014, Singapore [bib] @inproceedings{cui:2014:improvingDNN,
      Author = {Xiaodong Cui, Brian Kingsbury, Jia Cui, Bhuvana Ramabhadran, Andrew Rosenberg, Mohammed~Sadegh Rasooli, Owen Rambow, Nizar Habash, Vaibhava Goel},
      Booktitle = {Interspeech},
      Title = {Improving Deep Neural Network Acoustic Modeling For Audio Corpus Indexing Under The IARPA Babel Program},
      Year = {2014}}
  • Jia Cui, Bhuvana Ramabhadran, Xiaodong Cui, Andrew Rosenberg, Brian Kingsbury and Abhinav Sethy. Recent Improvements in Neural Network Acoustic Modeling for LVCSR in Low Resource Languages. Interspeech 2014, Singapore [bib] @inproceedings{cui:2014:recent,
      Author = {Jia Cui, Bhuvana Ramabhadran, Xiaodong Cui, Andrew Rosenberg, Brian Kingsbury, Abhinav Sethy},
      Booktitle = {Interspeech},
      Title = {Recent Improvements in Neural Network Acoustic Modeling for LVCSR in Low Resource Languages},
      Year = {2014}}
  • Justin Richards, Min Ma and Andrew Rosenberg. Using Word Burst Analysis to Rescore Keyword Search Candidates on Low-Resource Languages. ICASSP 2014, Florence, Italy [bib] @inproceedings{Soto:2014:rescoring,
      Author = {Justin Richards, Min Ma, Andrew Rosenberg},
      Booktitle = {ICASSP},
      Title = {Using Word Burst Analysis to Rescore Keyword Search Candidates on Low-Resource Languages},
      Year = {2014}}
  • Victor Soto, Erica Cooper, Lidia Mangu, Andrew Rosenberg and Julia Hirschberg. Rescoring Confusion Networks For Keyword Search. ICASSP 2014, Florence, Italy [bib] @inproceedings{Soto:2014:rescoring,
      Author = {Victor Soto, Erica Cooper, Lidia Mangu, Andrew Rosenberg and Julia Hirschberg},
      Booktitle = {ICASSP},
      Title = {Rescoring Confusion Networks For Keyword Search},
      Year = {2014}}
  • Helena Moniz, Ana Isabel Mata, Julia Hirschberg, Fernando Batistia, Andrew Rosenberg and Isabel Trancoso. Extending AuToBI to prominence detection in European Portuguese. Speech Prosody 2014, Dublin Ireland [bib] @inproceedings{Moniz:2014:autobi,
      Author = {Helena Moniz, Ana Isabel Mata, Julia Hirschberg, Fernando Batistia, Andrew Rosenberg and Isabel Trancoso},
      Booktitle = {Speech Prosody},
      Title = {Extending AuToBI to prominence detection in European Portuguese},
      Year = {2014}}

2013

  • Felix Grezes, Justin Richards, Andrew Rosenberg. Let Me Finish: Automatic Conflict Detection Using Speaker Overlap. Interspeech 2013, Lyon, France [bib] @inproceedings{Grezes:2013:conflict,
      Author = {Felix Grezes, Justin Richards, Andrew Rosenberg},
      Booktitle = {Interspeech},
      Title = {Let Me Finish: Automatic Conflict Detection Using Speaker Overlap},
      Year = {2013}}
  • Andrew Rosenberg. Modeling Prosodic Sequences with K-Means and Dirichlet Process GMMs. Interspeech 2013, Lyon, France [bib] @inproceedings{Rosenberg:2013:sequences,
      Author = {Andrew Rosenberg},
      Booktitle = {Interspeech},
      Title = {Modeling Prosodic Sequences with K-Means and Dirichlet Process GMMs},
      Year = {2013}}
  • Rachel Rakov, Andrew Rosenberg. "Sure, I Did the Right Thing": A System for Sarcasm Detection in Speech. Interspeech 2013, Lyon, France [bib] @inproceedings{Rakov:2013:sarcasm,
      Author = {Rachel Rakov, Andrew Rosenberg},
      Booktitle = {Interspeech},
      Title = {"Sure, I Did the Right Thing": A System for Sarcasm Detection in Speech},
      Year = {2013}}
  • David Guy Brizan, Guozhen An, Andrew Rosenberg. Detecting Laughter and Filled Pauses Using Syllable-based Features. Interspeech 2013, Lyon, France [bib] @inproceedings{Brizan:2013:laughter,
      Author = {David~Guy Brizan, Guozhen An, Andrew Rosenberg},
      Booktitle = {Interspeech},
      Title = {Detecting Laughter and Filled Pauses Using Syllable-based Features},
      Year = {2013}}
  • Victor Soto, Erica Cooper, Andrew Rosenberg, Julia Hirschberg. Cross-Language Phrase Boundary Detection. ICASSP 2013, Vancouver, Canada [bib] @inproceedings{Soto:2013:crosslanguage,
      Author = {Victor Soto, Erica Cooper, Andrew Rosenberg, Julia Hirschberg},
      Booktitle = {ICASSP},
      Title = {Cross-Lanugage Phrase Boundary Detection},
      Year = {2013}}

2012

  • Andrew Rosenberg. Rethinking the Corpus: Moving towards Dynamic Linguistic Resources. Interspeech 2012, Portland OR [bib] @inproceedings{Rosenberg:2012:rethinking,
      Author = {Andrew Rosenberg},
      Booktitle = {Interspeech},
      Title = {Rethinking the Corpus: Moving towards Dynamic Linguistic Resources},
      Year = {2012}}
  • Andrew Rosenberg. Classifying Skewed Data: Importance Weighting to Optimize Average Recall Interspeech 2012, Portland OR [bib] @inproceedings{Rosenberg:2012:classifying,
      Author = {Andrew Rosenberg},
      Booktitle = {Interspeech},
      Title = {Classifying Skewed Data: Importance Weighting to Optimize Average Recall},
      Year = {2012}}
  • Andrew Rosenberg, Raul Fernandez and Bhuvana Ramabhadran. Prosody Modeling for Phrase Boundary Assignment from Text in Multiple Domains. Interspeech 2012, Portland OR [bib] @inproceedings{Rosenberg:2012:phrase,
      Author = {Andrew Rosenberg and Raul Fernandez and Bhuvana Ramabhadran},
      Booktitle = {Interspeech},
      Title = {Prosody Modeling for Phrase Boundary Assignment from Text in Multiple Domains},
      Year = {2012}}
  • Andrew Rosenberg. Using Prominence and Phrasing Predictions to Improve Weighted Dictionary Pronunciation Models Interspeech 2012, Portland OR [bib] @inproceedings{Rosenberg:2012:prominence,
      Author = {Andrew Rosenberg},
      Booktitle = {Interspeech},
      Title = {Using Prominence and Phrasing Predictions to Improve Weighted Dictionary Pronunciation Models},
      Year = {2012}}
  • Sameer Maskey and Andrew Rosenberg. Power mean Pyramid Scores for Summarization Evaluation. Interspeech 2012, Portland OR [bib] @inproceedings{Maskey:2012:PowerMean,
      Author = {Sameer Maskey and Andrew Rosenberg},
      Booktitle = {Interspeech},
      Title = {Power mean Pyramid Scores for Summarization Evaluation},
      Year = {2012}}
  • Ian Kaplan and Andrew Rosenberg. Analysis of Speech Transcripts to Predict Winners of Presidential and Vice-Presidential Debates. IEEE SLT 2012, Miami FL [bib] @inproceedings{Kaplan:2012:analysis,
      Author = {Ian Kaplan and Andrew Rosenberg},
      Booktitle = {IEEE Workshop on Spoken Language Technology},
      Title = {Analysis of Speech Transcripts to Predict Winners of Presidential and Vice-Presidential Debates},
      Year = {2012}}
  • Andrew Rosenberg. Modeling Intensity Contours and the Interaction between Pitch and Intensity to Improve Automatic Prosodic Event Detection and Classification. IEEE SLT 2012, Miami FL [bib] @inproceedings{Rosenberg:2012:modeling,
      Author = {Andrew Rosenberg},
      Booktitle = {Interspeech},
      Title = {Modeling Intensity Contours and the Interaction between Pitch and Intensity to Improve Automatic Prosodic Event Detection and Classification},
      Year = {2012}}
  • Andrew Rosenberg, Erica Cooper, Rivka Levitan, Julia Hirschberg. Cross Language Prominence Detection. Speech Prosody 2012 [bib] @inproceedings{Rosenberg:2012:cross,
      Author = {Andrew Rosenberg and Erica Cooper and Rivka Levitan and Julia Hirschberg},
      Booktitle = {Speech Prosody},
      Title = {Cross Language Prominence Detection},
      Year = {2012}}

2011

  • Andrew Rosenberg. Using Mutual Information to Identify Regions of Analysis for Prosodic Analysis. Interspeech 2011 @inproceedings{Rosenberg:2011:mutual,
      Author = {Andrew Rosenberg},
      Booktitle = {Interspeech},
      Title = {Using Mutual Information to Identify Regions of Analysis for Prosodic Analysis},
      Year = {2011}}
    [pdf]
  • Andrew Rosenberg. Symbolic and Direct Sequential Modeling of Prosody for Classification of Speaking Style and Nativeness. Interspeech 2011 @inproceedings{Rosenberg:2011:symbolic,
      Author = {Andrew Rosenberg},
      Booktitle = {Interspeech},
      Title = {Symbolic and Direct Sequential Modeling of Prosody for Classification of Speaking-Style and Nativeness},
      Year = {2011}}
    [pdf]
  • Fadi Biadsy, William Wang, Andrew Rosenberg, Julia Hirschberg. Intoxication Detection using Phonetic, Phonotactic and Prosodic Cues. Interspeech 2011 @inproceedings{Biadsy:2011:intoxication,
      Author = {Fadi Biadsy and William Yang Wang and Andrew Rosenberg and Julia Hirschberg},
      Booktitle = {Interspeech},
      Title = {Intoxication Detection using Phonetic, Phonotactic and Prosodic Cues},
      Year = {2011}}
    [pdf]
  • Andrew Rosenberg, Raul Fernandez, Bhuvana Ramabhadran. "What is Dengue Fever..." Modeling and Predicting Pronunciation Errors in a Text-to-Speech System. Interspeech 2011 @inproceedings{Rosenberg:2011:dengue,
      Author = {Andrew Rosenberg and Raul Fernandez and Bhuvana Ramabhadran},
      Booktitle = {Interspeech},
      Title = {``What is Dengue Fever{\ldots}'' Modeling and Predicting Pronunciation Errors in a Text-to-Speech System},
      Year = {2011}}
    [pdf]
  • Matt Huenerfauth, Pengfei Lu, Andrew Rosenberg, Evaluating Importance of Facial Expression in American Sign Language and Pidgin Signed English Animations. The 13th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) @inproceedings{Huenerfauth:2011:evaluating,
      Author = {Matt Huenerfauth and Pengfei Lu and Andrew Rosenberg},
      Booktitle = {The 13th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS)},
      Title = {Evaluating Importance of Facial Expression in Americal Sign Language and Pidgin Signed English Animations},
      Year = {2011}}
    [pdf]
  • Ilnur Icke and Andrew Rosenberg. 2011. Multiobjective Genetic Programming for Visual Analytics. EuroGP 2011 [pdf]

2010

  • Andrew Rosenberg. 2010. AuToBI - A Tool for Automatic ToBI Annotation. INTERSPEECH 2010 [pdf]
  • Andrew Rosenberg. 2010. Classification of Prosodic Events using Quantized Contour Modeling. HLT-NAACL 2010 [pdf]
  • Andrew Rosenberg and Julia Hirschberg. 2010. Production of English Prominence by Native Mandarin Chinese Speakers. Workshop on Prosodic Prominence: Perceptual, Automatic Identification [pdf]
  • Andrew Rosenberg, Julia Hirschberg and Kim Manis. 2010. Perception of English Prominence by Native Mandarin Chinese Speakers. Speech Prosody 2010 [pdf]
  • Ilknur Icke and Andrew Rosenberg. 2010. Dimensionality Reduction using Symbolic Regression. GECCO 2010 [pdf]

2009

2008

  • M. Ostendorf, B. Favre, D. Grishman, M. Hakkani-Tur, M. Harper, D. Hillard, J. Hirschberg, H. Ji, J. G. Kahn, Y. Liu, S. Maskey, E. Matusov, H. Ney, A. Rosenberg, E. Shriberg, W. Wang, C. Wooters. 2008. Speech Segmentation and its Impact on Spoken Document Processing IEEE Signal Processing Magazine
  • Fadi Biadsy, Andrew Rosenberg, Rolf Carlson, Julia Hirschberg and Eva Strangert. 2008. A Cross-Cultural Comparison of American, Palestinian, and Swedish Perception of Charismatic Speech. Speech Prosody
  • Andrew Rosenberg and Julia Hirschberg. 2008. Charisma Perception from Text and Speech Speech Communication. 2008 [ScienceDirect]
  • Sameer Maskey, Andrew Rosenberg and Julia Hirschberg. 2008. Intonational Phrases for Speech Summarization INTERSPEECH '08

2007

2006-2004

Valid XHTML 1.0 Transitional