Publications

• PROFILE
• RESEARCH
• PUBLICATIONS

Working papers

• Takuya Hiraoka, Guanquan Wang, Takashi Onishi, Yoshimasa Tsuruoka.
  Which Experiences Are Influential for RL Agents? Efficiently Estimating The Influence of Experiences
  arXiv:2405.14629, 2024
  source code, demo video 1, demo video 2
  
• Takuya Hiraoka.
  Efficient Sparse-Reward Goal-Conditioned Reinforcement Learning with a High Replay Ratio and Regularization
  arXiv:2312.05787, 2023
  source code, demo video
  
• Takahisa Imagawa, Takuya Hiraoka, Yoshimasa Tsuruoka.
  Unsupervised Discovery of Continuous Skills on a Sphere
  arXiv preprint aarXiv:2305.14377, 2023
  
• Takuya Hiraoka, Masaaki Tsuchida, Yotaro Watanabe.
  Deep Reinforcement Learning for Inquiry Dialog Policies with Logical Formula Embeddings
  arXiv preprint arXiv:1708.00667, 2017
  

Journal papers

• Guanquan Wang, Takuya Hiraoka, Yoshimasa Tsuruoka.
  Planning with Consistency Models for Model-Based Offline Reinforcement Learning
  Transactions on Machine Learning Research (TMLR), 2024 (To appear)
  
• Takahisa Imagawa, Takuya Hiraoka, Yoshimasa Tsuruoka.
  Off-Policy Meta-Reinforcement Learning With Belief-Based Task Inference
  IEEE Access. Vol. 10, 2022
  
• Katsumi Hisao, Koichiro Yoshino, Takuya Hiraoka, Kosuke Akimoto, Kazeto Yamamoto, Shota Motoura, Kunihiko Sadamasa, Satoshi Nakamura.
  Optimization of Information-Seeking Dialogue Strategy for Argumentation-Based Dialogue System
  Transactions of the Japanese Society for Artificial Intelligence, Vol.35, No.1, 2020
  
• Takuya Hiraoka, Graham Neubig, Sakriani Sakti, Tomoki Toda, Satoshi Nakamura.
  Learning cooperative persuasive dialogue policies using framing
  Speech Communication, Vol. 84, pp. 83--96, Elsevier, 2016
  ScienceDirect
  
• Takuya Hiraoka, Kallirroi Georgila, Elnaz Nouri, David Traum, Satoshi Nakamura.
  Reinforcement Learning of Multi-Party Trading Dialog Policies
  Transactions of the Japanese Society for Artificial Intelligence, Vol. 31, No. 4, 2016
  

International conferences/workshops (refereed)

• Guanquan Wang, Takuya Hiraoka, Yoshimasa Tsuruoka.
  Planning with Consistency Models for Model-Based Offline Reinforcement Learning
  RLC 2024 Workshop on Deployable RL, 2024
  poster
• Guanquan Wang, Takuya Hiraoka, Yoshimasa Tsuruoka.
  Application of Reward Shaping in Industrial Manufacturing Reinforcement Learning Environment
  SICE Annual Conference, 2023
  
• Takuya Hiraoka, Takahisa Imagawa, Taisei Hashimoto, Takashi Onishi, Yoshimasa Tsuruoka.
  Dropout Q-Functions for Doubly Efficient Reinforcement Learning
  The Tenth International Conference on Learning Representations (ICLR), 2022
  poster, slide, arXiv version, source code
  
• Takuya Hiraoka, Takahisa Imagawa, Voot Tangkaratt, Takayuki Osa, Takashi Onishi, Yoshimasa Tsuruoka.
  Meta-Model-Based Meta-Policy Optimization
  The 13th Asian Conference on Machine Learning (ACML), 2021
  poster, video presentation, demo video, arXiv version, source code
  
• Taisei Hashimoto, Takashi Onishi, Takuya Hiraoka, Yoshimasa Tsuruoka.
  Controlling the Production Load of a Vinyl Acetate Monomer Plant by Deep Reinforcement Learning
  SICE Annual Conference, 2021
  
• Takahisa Imagawa, Takuya Hiraoka, Yoshimasa Tsuruoka.
  Off-Policy Meta-Reinforcement Learning Based on Feature Embedding Spaces
  ICML 2019 Workshop on Lifelong Learning Workshop (LifelongML), 2020
  video presentation
  
• Takuya Hiraoka, Takahisa Imagawa, Tatsuya Mori, Takashi Onishi, Yoshimasa Tsuruoka.
  Learning Robust Options by Conditional Value at Risk Optimization
  Thirty-third Conference on Neural Information Processing System (NeurIPS), 2019
  poster, arXiv version, source code, demo video
  
• Kosuke Akimoto, Takuya Hiraoka, Kunihiko Sadamasa, Mathias Niepert.
  Cross-Sentence N-ary Relation Extraction using Lower-Arity Universal Schemas
  Conference on Empirical Methods in Natural Language Processing (EMNLP), 2019
  source code, dateaset
  
• Seydou Ba, Takuya Hiraoka, Takashi Onishi, Toru Nakata, Yoshimasa Tsuruoka.
  Monte Carlo Tree Search with Scalable Simulation Periods for Continuously Running Tasks
  31st International Conference on Tools with Artificial Intelligence (ICTAI), 2019
  arXiv version, source code
  
• Takahisa Imagawa, Takuya Hiraoka, Yoshimasa Tsuruoka.
  Optimistic Proximal Policy Optimization
  ICML 2019 Workshop on Exploration in RL (ERL 2019), 2019
  
• Hisao Katsumi, Takuya Hiraoka, Koichiro Yoshino, Kazeto Yamamoto, Shota Motoura, Kunihiko Sadamasa and Satoshi Nakamura.
  Optimization of Information-Seeking Dialogue Strategy for Argumentation-Based Dialogue System
  AAAI 2019 Workshop on Reasoning and Learning for Human-Machine Dialogues (DEEP-DIAL 2019), 2019
  arXiv version
  
• Takuya Hiraoka, Takashi Onishi, Yoshimasa Tsuruoka.
  Refining Manually-Designed Symbol Grounding and High-Level Planning by Policy Gradients
  IJCAI-ECAI 2018 Workshop on Learning & Reasoning (L&R), 2018
  arXiv version, press release (in Japanese)
  
• Seydou Ba, Takuya Hiraoka, Takashi Onishi, Toru Nakata, Yoshimasa Tsuruoka.
  Monte Carlo Tree Search with Scalable Simulation Periods for Continuously Running Tasks
  ICML / IJCAI / AAMAS 2018 Workshop on Planning and Learning (PAL-18), 2018
  
• Takuya Hiraoka, Shota Motoura, Kunihiko Sadamasa.
  Detecticon: A Prototype Inquiry Dialog System
  9th International Workshop on Spoken Dialog Systems (IWSDS), 2018
  Demo video in Japanese (with English subtitles), Presentation slides
  
• Yuiko Tsunomori, Graham Neubig, Takuya Hiraoka, Masahiro Mizukai, Sakriani Sakti, Tomoki Toda, Satoshi Nakamura.
  A Dialog System to Detect Deception
  7th International Workshop on Spoken Dialog Systems (IWSDS), 2016
  
• Takuya Hiraoka, Graham Neubig, Koichiro Yoshino, Tomoki Toda, Satoshi Nakamura.
  Active Learning for Example-based Dialog Systems
  7th International Workshop on Spoken Dialog Systems (IWSDS), 2016
  Source code
• Koichiro Yoshino, Takuya Hiraoka, Graham Neubig, Satoshi Nakamura.
  Dialogue State Tracking using Long Short Term Memory Neural Networks
  7th International Workshop on Spoken Dialog Systems (IWSDS), 2016
  Source code
• Takuya Hiraoka, Kallirroi Georgila, Elnaz Nouri, David Traum, Satoshi Nakamura.
  Reinforcement Learning in Multi-Party Trading Dialog
  The 16th Annual SIGdial Meeting on Discourse and Dialogue (SIGDIAL), 2015
  Source code, video recordings and slides
• Takuya Hiraoka, Graham Neubig, Sakriani Sakti, Tomoki Toda, Satoshi Nakamura.
  Evaluation of a Fully Automatic Cooperative Persuasive Dialogue System
  The 6th International Workshop on Spoken Dialog Systems (IWSDS), 2015
  Demo video in Japanese (with English subtitles), e-book version
  
• Riki Yoshida, Takuya Hiraoka, Graham Neubig, Sakriani Sakti, Tomoki Toda, Satoshi Nakamura.
  Unnecessary Utterance Detection for Avoiding Digressions in Discussion
  Asia Pacific Signal and Information Processing Association (APSIPA), 2014
  
• Takuya Hiraoka, Graham Neubig, Sakriani Sakti, Tomoki Toda, Satoshi Nakamura.
  Reinforcement Learning of Cooperative Persuasive Dialogue Policies using Framing
  The 25th International Conference on Computational Linguistics (COLING), 2014
  
• Takuya Hiraoka, Graham Neubig, Sakriani Sakti, Tomoki Toda, Satoshi Nakamura.
  Construction and Analysis of a Persuasive Dialogue Corpus
  The 5th International Workshop on Spoken Dialog Systems (IWSDS), 2014
  e-book version
  
• Takuya Hiraoka, Yuki Yamauchi, Graham Neubig, Sakriani Sakti, Tomoki Toda, Satoshi Nakamura.
  Dialogue Management for Leading the Conversation in Persuasive Dialogue Systems
  Automatic Speech Recognition and Understanding Workshop (ASRU), 2013
  Demo video in Japanese

Domestic conferences in Japan

• Yukiko Tanaka, Takuya Hiraoka, Yoshimasa Tsuruoka
  Learning Interpretable Control Policies with Decision Trees via the Cross-Entropy Method (In Japanese)
  Annual Conference on the Japanese Society for Artificial Intelligence (JSAI), 2019
• Hisao Katsumi, Takuya Hiraoka, Shota Motoura, Kazeto Yamamoto, Kunihiko Sadamasa, Koichiro Yoshino, Satoshi Nakamura.
  深層強化学習を用いた論証構築のための情報探索対話戦略の最適化 (In Japanese)
  RSJ, 2018
• Yukiko Tanaka, Takuya Hiraoka, Takashi Onishi, Yoshimasa Tsuruoka, Shohei Takino, Yosuke Matsumoto
  Toward Supporting Decision Making for Flood Control in a Hydroelectric Dam by Reinforcement Learning and Symbol Grounding (In Japanese)
  Annual Conference on the Japanese Society for Artificial Intelligence (JSAI), 2018
• Hisao Katsumi, Takuya Hiraoka, Shota Motoura, Kazeto Yamamoto, Kunihiko Sadamasa, Koichiro Yoshino, Satoshi Nakamura.
  論証構築のための情報探索対話戦略の最適化 (In Japanese)
  ANLP, 2018
• Yoko Ishikawa, Takuya Hiraoka, Masahiro Mizukami, Koichiro Yoshino, Graham Neubig, Satoshi Nakamura.
  対話状態推定のための外部知識ベースを利用した意味的素性の提案 (In Japanese)
  SIG-SLP, 2015. Student poster award
• Yuiko Tsunomori, Graham Neubig, Sakriani Sakti, Takuya Hiraoka, Masahiro Mizukami, Tomoki Toda, Satoshi Nakamura.
  A Dialog System to Detect Deception (In Japanese)
  SIG-SLUD, 2015
• Takuya Hiraoka, Graham Neubig, Sakriani Sakti, Tomoki Toda, Satoshi Nakamura.
  Construction and Evaluation of a Cooperative Persuasive Dialog System with Natural Language Understanding and Natural Language Generation
  The 29th Annual Conference of the Japanese Society for Artificial Intelligence (JSAI), 2015
  Demo video in Japanese
  
• Takuya Hiraoka, Graham Neubig, Sakriani Sakti, Tomoki Toda, Satoshi Nakamura.
  Reinforcement Learning of Cooperative Persuasive Dialogue Policies using Framing (Japanese)
  Autumn Conference on Acoustical Society of Japan (ASJ), 2014
• Takuya Hiraoka, Graham Neubig, Sakriani Sakti, Tomoki Toda, Satoshi Nakamura.
  Construction and Analysis of a Persuasive Dialogue Corpus (In Japanese)
  The 95th meeting of IPSJ Special Interest Group on Spoken Language Processing (SIG-SLP), 2013
• Takuya Hiraoka, Yuki Yamauchi, Graham Neubig, Sakriani Sakti, Tomoki Toda, Satoshi Nakamura.
  Persuasive Dialogue System with Dialogue Manager and Natural Language Generation Considering Guiding Users to Target Topics
  Annual Conference on the Japanese Society for Artificial Intelligence (JSAI), 2013
• Takuya Hiraoka, Graham Neubig, Sakriani Sakti, Tomoki Toda, Satoshi Nakamura.
  Analysis of a Dialogue Corpus for Construction of Persuasive Dialogue Systems
  Spring Conference on Acoustical Society of Japan (ASJ), 2013
• Takuya Hiraoka, Graham Neubig, Sakriani Sakti, Tomoki Toda, Satoshi Nakamura.
  Dialog Management Based on Guiding Users to New Topics in Persuasive Dialogue Systems
  The 95th meeting of IPSJ Special Interest Group on Spoken Language Processing (SIG-SLP), 2012
• Takuya Hiraoka, Graham Neubig, Sakriani Sakti, Tomoki Toda, Satoshi Nakamura.
  Examination for the Dialogue Management for the Persuasive Dialogue System
  Autumn Conference on Acoustical Society of Japan (ASJ), September 2012

Books

• Takuya Hiraoka, Shota Motoura, Kunihiko Sadamasa.
  Detecticon: A Prototype Inquiry Dialog System
  9th International Workshop on Spoken Dialog Systems (IWSDS), pp. 381--387, Springer, 2019
  
• Takuya Hiraoka, Graham Neubig, Yoshino Koichiro, Tomoki Toda, Satoshi Nakamura.
  Active learning for example-based dialog systems
  Dialogues with Social Robots. Springer Singapore, pp. 67--78, Springer, 2017
  
• Takuya Hiraoka, Graham Neubig, Sakriani Sakti, Tomoki Toda, Satoshi Nakamura.
  Construction and analysis of a persuasive dialogue corpus
  Situated Dialog in Speech-Based Human-Computer Interaction, pp. 125--138, Springer, 2016
  
• Takuya Hiraoka, Graham Neubig, Sakriani Sakti, Tomoki Toda, Satoshi Nakamura.
  Evaluation of a Fully Automatic Cooperative Persuasive Dialogue System
  Natural Language Dialog Systems and Intelligent Assistants, pp. 153--167, Springer, 2015