Yan-Ying Chen
Contact:
yanying ﹫ gmail ● com
My interest lies at the intersection of computer vision, natural language processing and applied machine learning. I worked on image/video to text generation, business activity/location recognition, visual/text sentiment analysis and recently focus on free-text report generation for medical images.
- Experiences
-
I am currently a Senior Research Scientist at FX Palo Alto Laboratory (FXPAL) where I joined as a Research Scientist in 2014. Before that I was a Visiting Researcher at Columbia University (Host: Prof. Shih-Fu Chang). I was fortunate to work with talented researchers in academia and industry, and published over 30 papers and US patents. See Google Scholar, DBLP and US Patent for more details.
- Education
-
Ph.D. in Computer Science
National Taiwan University, Taiwan
Advisor: Prof. Winston H. Hsu and Dr. Mark Liao - News
-
2019
A co-authored paper about medical image report generation is accepted by BMVC 2019.
A co-authored paper about abstractive title generation is accepted by ACL 2019.
A co-authored paper about knowledge tracing is accepted by WWW 2019.
2018
A co-authored paper about summarization for online conversations is accepted by EMNLP 2018
A co-authored paper about conversation disentanglement is accepted by NAACL 2018.
A co-authored paper about image localization is accepted by ICRA 2018.
Recent Proejects
Medical Image Report Generation, 2018 - present, at FXPAL
Adapted context-aware image to text generation to medical domains. Proposed the model that addressed the data bias problems in medical image reports and the need of higher accuracy on the descriptions of abnormalities. Lead the research and communication with collaborators.
Business Activity/Location Recognition and Image/Video Captioning, 2015 - 2018, at FXPAL
Leveraged deep learning technologies for identifying business venues and customer purchase stage from social media images and posts, and generating image descriptions relevant to business activities therein. Designed models that integrate context cross domains/tasks to improve and enrich image/video description generation.
Viewer Affective Comment Generation, 2013 - 2014, at Columbia University
Leveraged visual and text sentiment analysis to discover the relations between image content and the evoked viewer affects in its comments and learn an affective comment generator for social media images. [Project page]
Selected Publications
"Addressing Data Bias Problems for Chest X-ray Image Report Generation," Philipp Harzig, Yan-Ying Chen, Francine Chen, Rainer Lienhart, BMVC 2019
"Adversarial Domain Adaptation Using Artificial Titles for Abstractive Title Generation," Francine Chen, Yan-Ying Chen, ACL 2019
"ContextualNet: Exploiting Contextual Information Using LSTMs to Improve Image-Based Localization," Mitesh Patel, Brendan Emery, Yan-Ying Chen, ICRA 2018
"Video to Text Summary: Joint Video Summarization and Captioning with Recurrent Neural Networks," Sirius Chen, Yan-Ying Chen, Francine Chen, BMVC 2017.
"Ranking Convolutional Recurrent Neural Networks for Purchase Stage Identification on Imbalanced Twitter Data," Heike Adel, Francine Chen, Yan-Ying Chen, EACL 2017.
"Business-Aware Visual Concept Discovery from Social Media for Multimodal Business Venue Recognition," Bor-Chun Chen, Yan-Ying Chen, Francine Chen, Dhiraj Joshi, AAAI 2016.
"Using Business-Aware Latent Topics For Image Captioning In Social Media," Yan-Ying Chen, Francine Chen, Matthew Cooper and Dhiraj Joshi, IEEE ICME 2016.
"Assistive Image Comment Robot - A Novel Mid-Level Concept-Based Representation," Yan-Ying Chen, Tao Chen, Taikun Liu, Hong-Yuan Mark Liao, Shih-Fu Chang, IEEE Transactions on Affective Computing. 2015
"Visually Interpreting Names as Demographic Attributes by Exploiting Click-Through Data," Yan-Ying Chen, Yin-Hsi Kuo, Chun-Che Wu, Winston H. Hsu, AAAI 2015.
"Predicting Viewer Affective Comments Based on Image Content in Social Media," Yan-Ying Chen, Tao Chen, Winston H. Hsu, Hong-Yuan Mark Liao, Shih-Fu Chang, ACM ICMR 2014.
"Travel Recommendation by Mining People Attributes and Travel Group Types from Community-Contributed Photos," Yan-Ying Chen, An-Jung Cheng, Winston H. Hsu, IEEE Transactions on Multimedia, 2013.
"Discovering Informative Social Subgraphs and Predicting Pairwise Relationships from Group Photos," Yan-Ying Chen, Winston H. Hsu, Hong-Yuan Mark Liao, ACM Multimedia 2012.
"Learning Facial Attributes by Crowdsourcing in Social Media," Yan-Ying Chen, Winston H. Hsu, Hong-Yuan Mark Liao, WWW 2011.
More Publications
"Augmenting Knowledge Tracing by Considering Forgetting Behavior," Koki Nagatani, Qian Zhang, Masahiro Sato, Yan-Ying Chen, Francine Chen, Tomoko Ohkuma, WWW 2019
"Harnessing Popularity in Social Media for Extractive Summarization of Online Conversations," Ryuji Kano, Yasuhide Miura, Motoki Taniguchi, Yan-Ying Chen, Francine Chen, Tomoko Ohkuma, EMNLP 2018
"Learning to Disentangle Interleaved Conversational Threads with a Siamese Hierarchical Network and Similarity Ranking," Jyun-Yu Jiang, Francine Chen, Yan-Ying Chen, Wei Wang, NAACL-HLT 2018
"Image-Based User Profiling of Frequent and Regular Venue Categories," Ryosuke Shigenaka, Yan-Ying Chen, Francine Chen, Dhiraj Joshi, Yukihiro Tsuboshita, IEEE ICME 2017.
"Scalable Face Track Retrieval in Video Archives using Bag-of-Faces Sparse Representation," Bor-Chun Chen, Yan-Ying Chen, Yin-Hsi Kuo, Thanh Duc Ngo, Duy-Dinh Le, Shin'ichi Satoh, Winston H. Hsu, IEEE Transactions on Circuits and Systems for Video Technology, 2017.
"Recurrent Neural Networks for Customer Purchase Prediction on Twitter," Mandy Korpusik, Shigeyuki Sakaki, Francine Chen, Yan-Ying Chen, Workshop at RecSys 2016.
"Corpus for Customer Purchase Behavior Prediction in Social Media," Shigeyuki Sakaki, Francine Chen, Mandy Korpusik and Yan-Ying Chen, LREC 2016.
"Identify Visual Human Signature in Community via Wearable Camera," Chia-Chin Tsao, Yan-Ying Chen, Yu-Lin Hou, Winston H. Hsu, ICASSP 2015.
"Building User Profiles from Shared Photos," Dhiraj Joshi, Matthew Cooper, Francine Chen, Yan-Ying Chen, ACM Multimedia Workshop (Multimedia Commons) 2015.
"Object-Based Visual Sentiment Concept Analysis and Application," Tao Chen, Felix X. Yu, Jiawei Chen, Yin Cui, Yan-Ying Chen, Shih-Fu Chang, ACM Multimedia 2014.
"Discovering the City by Mining Diverse and Multimodal Data Streams," Yin-Hsi Kuo, Yan-Ying Chen, Bor-Chun Chen, Wen-Yu Lee, Chun-Che Wu, Chia-Hung Lin, Yu-Lin Hou, Wen-Feng Cheng, Yi-Chih Tsai, Chung-Yen Hung, Liang-Chi Hsieh, Winston Hsu, ACM Multimedia 2014. (Grand Challenge Finalist)
"Facial Attribute Space Compression by Latent Human Topic Discovery," Chia-Hung Lin, Yan-Ying Chen, Bor-Chun Chen, Yu-Lin Hou, Winston H. Hsu, ACM Multimedia 2014.
"Automatic Facial Image Annotation and Retrieval by Integrating Voice Label and Visual Appearance," Hong-Wun Jheng, Bor-Chun Chen, Yan-Ying Chen, Winston H. Hsu, ACM Multimedia 2014.
"Enabling Low Bitrate Mobile Visual Recognition - A Performance versus Bandwidth Evaluation," Yu-Chuan Su, Tzu-Hsuan Chiu, Yan-Ying Chen, Chun-Yen Yeh, Winston H. Hsu, ACM Multimedia 2013.
"Search-Based Relevance Association with Auxiliary Contextual Cues," Chun-Che Wu, Kuan-Yu Chu, Yin-Hsi Kuo, Yan-Ying Chen, Wen-Yu Lee, Winston H. Hsu, ACM Multimedia 2013. (Grand Challenge Multimodal Award)
"Automatic Training Image Acquisition and Effective Feature Selection from Community-Contributed Photos for Facial Attribute Detection," Yan-Ying Chen, Winston H. Hsu, Hong-Yuan Mark Liao, IEEE Transactions on Multimedia, 2013.
"Scalable Face Image Retrieval using Attribute-Enhanced Sparse Codewords," Bor-Chun Chen, Yan-Ying Chen, Yin-Hsi Kuo, Winston H. Hsu, IEEE Transactions on Multimedia, 2013.
"People Search and Activity Mining in Large-Scale Community-Contributed Photos," Yan-Ying Chen, Winston H. Hsu, Hong-Yuan Mark Liao, ACM Multimedia 2012. [Doctoral Symposium Best Paper Award]
"Where is Who: Large-Scale Photo Retrieval by Facial Attributes and Canvas Layout," Yu-Heng Lei, Yan-Ying Chen, Bor-Chun Chen, Lime Iida, Winston Hsu, ACM SIGIR 2012.
"Learning by Expansion: Exploiting Social Media for Image Classification with Few Training Examples," Sheng-Yuan Wang, Wei-Shing Liao, Liang-Chi Hsieh, Yan-Ying Chen, Winston H. Hsu, Neurocomputing, 2012.
"Personalized Travel Recommendation by Mining People Attributes from Community-Contributed Photos," An-Jung Cheng, Yan-Ying Chen, Yen-Ta Huang, Winston H. Hsu, Hong-Yuan Mark Liao, ACM Multimedia 2011.
"Semi-Supervised Face Image Retrieval Using Sparse Coding with Identity Constraint," Bor-Chun Chen, Yin-His Kuo, Yan-Ying Chen, Kuan-Yu Chu, Winston H. Hsu, ACM Multimedia 2011.
"Photo Search by Face Positions and Facial Attributes on Touch Devices," Yu-Heng Lei, Yan-Ying Chen, Bor-Chun Chen, Hsiao-Hang Su, Lime Iida, Winston Hsu, ACM Multimedia 2011. [First Prize in ACM Multimedia Grand Challenge 2011]
Awards
FXPAL Significant Achievement Award 2018, 2019
Final List of Best Paper Award in IEEE ICME 2017
Multimodal Award in ACM Multimedia Grand Challenge 2014
IICM Ph.D. Dissertation Award, 2014
TAAI Ph.D. Dissertation Award, 2014
First Place in Microsoft Research - Bing Image Retrieval Challenge 2013
Multimodal Award in ACM Multimedia Grand Challenge 2013
2012 ACM Multimedia Doctoral Symposium Best Paper Award
WWW 2011 Google Fellowship for Women
First Prize in ACM Multimedia Grand Challenge 2011
2013 Study Abroad Program sponsored by National Science Council, Taiwan
2012 Summer Visit Program to Japan sponsored by Interchange Association, Japan
2nd Prize in Chunghwa Telecom Innovation Awards 2011
2011 Outstanding Students Conference Travel Grant from Foundation for the Advancement of Outstanding Scholarship