Bio

I am a Ph.D. Student in the Institute of Artificial Intelligence and Robotics (IAIR) at Xi'an Jiaotong University, supervised by Prof. Ping Wei. I received my M.S. degree from Huazhong University of Science and Technology, supervised by Assoc. Prof. Gang Peng.

I focus on multimodal learning and video understanding, which utilizes various modalities (e.g., image, audio, text, etc) to understand long-term untrimmed videos, supporting downstream tasks like video moment retrieval, highlight detection, robot perception, etc. I aim to develop multimodal learning from the perspective of human brain intelligence/cognitive science, with the goal of emulating the brain's cognitive reasoning process. Since the human brain is the most marvelous organ in the universe. Currently, I am investigating the zero-shot potential of multimodal large language models(MLLMs). I aspire to develop AI technologies that genuinely benefit society.

My research interests include: (1) Multimodal Learning (2) Video Understanding (3) Multimodal Large Language Models (4) Robot (5) Deep Reinforcement Learning

Publications

    Accepted/Preprint/In Submission/Patents
    For more research, please visit the Google Scholar.

    2024
  1. Task-Driven Exploration: Decoupling and Inter-Task Feedback for Joint Moment Retrieval and Highlight Detection
    Jin Yang, Ping Wei, Huan Li, Ziyang Ren
    CVPR 2024
  2. Learning Scene-Goal-Aware Motion Representation for Trajectory Prediction
    Ziyang Ren, Ping Wei, Haowen Tang, Huan Li, Jin Yang
    BMVC 2024
  3. Cross Time-Frequency Transformer for Temporal Action Localization
    Jin Yang, Ping Wei, Nanning Zheng
    IEEE Transactions on Circuits and Systems for Video Technology (TCSVT 2024)
  4. In Submission
    Perceptual Consistency-Driven Abstraction via Minimax Optimization for Joint Moment Retrieval and Highlight Detection
    Jin Yang, Ping Wei, Huan Li, Nanning Zheng
    IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI 2024)
  5. In Submission
    Learning Unified Patterns of Multimodalties for Joint Moment Retrieval and Highlight Detection
    Jin Yang, Ping Wei
    IEEE Transactions on Image Processing (TIP 2024)
  6. In Submission
    Asymmetric Apparent-aware Relation Consistency Learning for Video Relation Grounding
    Huan Li, Ping Wei, Jin Yang, Nanning Zheng
    IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI 2024)
  7. 2023
  8. Gated Multi-Scale Transformer for Temporal Action Localization
    Jin Yang, Ping Wei, Ziyang Ren, Nanning Zheng
    IEEE Transactions on Multimedia (TMM 2023)
  9. 2022
  10. Deep reinforcement learning with a stage incentive mechanism of dense reward for robotic trajectory planning
    Gang Peng, Jin Yang*, Xinde Li, Mohammad Omar Khyam ( * student first author)
    IEEE Transactions on Systems, Man, and Cybernetics: Systems (TSMC 2022)
  11. 2021
  12. DDPG with meta-learning-based experience replay separation for robot trajectory planning
    Jin Yang, Gang Peng
    ICCAR 2021
    Oral presentation
Patent
Projects

Selected projects from 2019 to present. You can click on the links to view details.

Education/Work Experiences
Research/Work
Education
Selected Honors & Awards