MediaBrain
Shanghai Jiao Tong University
Cooperative Medianet Innovation Center
Shanghai Jiao Tong University
Email: xiaoyun.zhang@sjtu.edu.cn
No. 5 EIEE Building East 303A
800 Dongchuan Rd. Minhang
Shanghai, 200240, China
TEL: +86-21-34205012
FAX: +86-21-34204155
Xiaoyun Zhang (张小云) Ph.D
Xiaoyun Zhang, Professor of Cooperative Medianet Innovation Center (CMIC), Shanghai Jiao Tong University. She received her Ph.D (supervised by Yuncai Liu) on Pattern Recognition from Shanghai Jiao Tong University and Master (supervised by Zongben Xu) on Applied Mathematics from Xi`an Jiao Tong University respectively. She has co-authorized papers on IEEE TPAMI, TIP, CVPR, ICCV, etc., and her Ph.D. thesis has been nominated as “National 100 Best Ph.D. Theses of China”. She has been a visiting scholar of Harvard University for one year. She is also a member of State Key Laboratory of UHD video and audio production and presentation, and has collaboration with CCTV in video restoration and enhancement.
MediaBrain group of CMIC currently has 7 professors including from Oxford, Cambridge, CMU, focusing on multimedia and medical area. Our group provides compute resource with over hundreds of GPUs, and experienced writing guide for top level research papers. Currently, our group has over 80 master and Ph.d students with good academic atmosphere and rich activities, including cutting-edge seminars for academic, and birthday celebration for fun.
Research
Her research interests include
-
- image/video processing and enhancement, including super-resolution, face restoration, and video interpolation, etc.
- deep generative model and its application in image and video generation
- pretrained and multi-modal large model, and its application in the emerging AI Generated Content (AIGC)
Publications
Journal
- Chen Ju, Peisen Zhao, Siheng Chen, Ya Zhang, Xiaoyun Zhang, Qi Tian，Adaptive Mutual Supervision for Weakly-Supervised Temporal Action Localization， Accepted by TMM,2022.
- Guo Lu, Xiaoyun Zhang, Wanli Ouyang, Li Chen, Zhiyong Gao, Dong Xu, “An End-to-End Learning Framework for Video Compression,” IEEE Transactions on Pattern Analysis and Machine Intelligence，43, no.10, pp.3292-3308, Oct., 2021.
- Wenbo Bao, Wei-Sheng Lai, Xiaoyun Zhang, Zhiyong Gao, Ming-Hsuan Yang, “MEMC-Net: Motion Estimation and Motion Compensation Driven Neural Network for Video Interpolation and Enhancement,” IEEE Transactions on Pattern Analysis and Artificial Intelligence, vol. 43, no. 3, pp.933-948, March 2021.
- Shixiang Feng, Beibei Liu, Ya Zhang, Xiaoyun Zhang, Yuehua Li，”Two-Stream Compare and Contrast Network for Vertebral Compression Fracture Diagnosis”, IEEE Transactions on Medical Imaging, 40(9):2496-2506, 2021.
- Chaofan Ma，Qisen Xu，Xiangfeng Wang，Bo Jin，Xiaoyun Zhang，Yanfeng Wang，Ya Zhang, “Boundary-aware Supervoxel-level Iteratively Refined Interactive 3D Image Segmentation with Multi-agent Reinforcement Learning”, IEEE Transactions on Medical Imaging, 40(10):2563-2574, 2021.
- Guo Lu, Xiaoyun Zhang, Wanli Ouyang, Dong Xu, Li Chen, Zhiyong Gao, ” Deep Non-local Kalman Network for Video Compression Artifact Reduction,” IEEE Transaction on Image Processing, vol.29, pp. 1725-1737, 2020.
- Chunlei Cai, Li Chen, Xiaoyun Zhang, Zhiyong Gao, “End-to-End Optimized ROI Image Compression,” IEEE Transactions on Image Processing, 29, pp. 3442-3457, 2020.
- Qiang Hu, Jun Zhou, Xiaoyun Zhang, Zhiyong Gao, Ming-ting Sun, “In-loop perceptual model-based rate-distortion optimization for HEVC real-time encoder,” Journal of Real-Time Image Processing, 2020, 17(2): 293-311.
- Qiang Hu, Jun Zhou, Xiaoyun Zhang, Zhiru Shi, Zhiyong Gao, “Viewport-Adaptive 360-degree Video Coding,” Multimedia Tools and Applications, Oct. 2019.
- Wenbo Bao, Xiaoyun Zhang, Li Chen, Zhiyong Gao, “KalmanFlow 2.0: Efficient Video Optical Flow Estimation via Context-Aware Kalman Filtering,” IEEE Transaction on Image Processing, vol.28, no.9, pp. 4233 – 4246, Sept. 2019.
- Chunlei Cai, Li Chen, Xiaoyun Zhang, Zhiyong Gao, “Efficient Variable Rate Image Compression with Multi-scale Decomposition Network,” IEEE Transactions on Circuits and Systems for Video Technology, vol.29, no.12, pp. 3687-3700, Dec. 2019.
- Wenbo Bao, Xiaoyun Zhang, Li Chen, Lianghui Ding, Zhiyong Gao, “High-Order Model and Dynamic Filtering for Frame Rate Up-Conversion, ” IEEE Transaction on Image Processing, vol. 27, no. 8, pp. 3813-3826, Aug, 2018.
- Guo Lu, Xiaoyun Zhang, Li Chen, Zhiyong Gao, “Novel Integration of Frame Rate Up Conversion and HEVC Coding Based on Rate-Distortion Optimization,” IEEE Transaction on Image Processing, vol. 27, no. 2, pp. 678-691, Feb. 2018.
- Bing Yang, Xiaoyun Zhang, Li Chen, Zhiyong Gao, “Spatiotemporal salient object detection based on distance transform and energy optimization,” Neurocomputing, vol. 266, pp. 165-175, 2017.
- Bing Yang, Xiaoyun Zhang, Li Chen, Hua Yang, Zhiyong Gao, “Edge Guided Salient Object Detection,” Neurocomputing, vol. 221, pp. 60-71, 2017.
- Yong Guo, Li Chen, Zhiyong Gao, Xiaoyun Zhang, “Frame Rate Up-Conversion Using Linear Quadratic Motion Estimation and Trilateral Filtering Motion Smoothing,” Journal of Display Technology, vol. 12, no.1, pp. 89-98, Jan. 2016.
- Qiang Hu, Xiaoyun Zhang, Zhiru Shi, Zhiyong Gao, “Neyman-Pearson Based Early Mode Decision for HEVC Encoding,” IEEE Transactions on Multimedia, vol. 18, no. 3, pp. 379-391, Mar. 2016.
- Bing Yang, Xiaoyun Zhang, Li Chen, Zhiyong Gao, “Principal Component Analysis-based Visual Saliency Detection,” IEEE Transactions on Broadcasting, 62, no. 4, pp.842-854, Dec 2016.
- Yong Guo, Li Chen, Zhiyong Gao, Xiaoyun Zhang, “Frame Rate Up-Conversion Method for Video Processing Applications,” IEEE Transactions on Broadcasting, vol. 60, no. 4, pp. 659-669, Dec. 2014.
- Chen Gang, Yang Bing, Zhang Xiaoyun, Gao Zhiyong，Complexity control algorithm based on adaptive mode selection for interframe coding in high efficiency video coding，Journal of Electronic Imaging, 2017.
Conference (recent 5 years)
- Zhixin Wang, Xiaoyun Zhang, Ziying Zhang, Huangjie Zheng, Mingyuan Zhou, Ya Zhang, Yanfeng Wang. DR2: Diffusion-based Robust Degradation Remover for Blind Face Restoration, accepted by CVPR2023.
- Qinye Zhou, Ziyi Li, Weidi Xie, Xiaoyun Zhang, Ya Zhang, Yanfeng Wang. A Simple Plugin for Transforming Images to Arbitrary Scales[J]. British Machine Vision Virtual Conference 2022.
- Yixuan Huang, Xiaoyun Zhang, et al., Task Decoupled Framework for Reference-based Super-Resolution, CVPR 2022.
- Baisong Guo, Xiaoyun Zhang, et al., LAR-SR: A Local Autoregressive Model for Image Super-Resolution, CVPR 2022.
- Yangyi Dong, Xiaoyun Zhang, et c., Unpaired Face Restoration via Learnable Cross-Quality Shift, CVPR 2022 NTIRE.
- Tianyue Cao, Lianyu Du, Xiaoyun Zhang, Siheng Chen, Ya Zhang, Yan-Feng Wang, “CaT: Weakly Supervised Object Detection with Category Transfer”, in ICCV2021, pp. 3070-3079.
- Lianyu Du, Liwei Hu, Xiaoyun Zhang, Yumin Zhong, Ya Zhang, Yanfeng Wang, “Unsupervised Segmentation Framework with Active Contour Models for Cine Cardiac MRI”, in ICIP 2021.
- Xingyue Pu, Tianyue Cao, Xiaoyun Zhang, Xiaowen Dong, Siheng Chen, “Learning to learn graph topologies”, NeurIPS 2021.
- Guo Lu, Chunlei Cai, Xiaoyun Zhang, Li Chen, Wanli Ouyang, Dong Xu，Zhiyong Gao, “Content adaptive and error propagation aware deep video compression,” in European Conference on Computer Vision (ECCV), pp.456-472, Springer, Cham, 2020.
- Yingying Xue, Shixiang Feng, Ya Zhang, Xiaoyun Zhang, Yanfeng Wang, “Dual-task Self-supervision for Cross-Modality Domain Adaptation”, MICCAI 2020, pp. 408-417.
- Xuan Liao, Wenhao Li, Qisen Xu, Xiangfeng Wang, Bo Jin, Xiaoyun Zhang, Yanfeng Wang , Ya Zhang, “Iteratively-Refined Interactive 3D Medical Image Segmentation with Multi-Agent Reinforcement Learning”, CVPR 2020, pp. 9394-9402.
- Chunlei Cai, Li Chen, Xiaoyun Zhang, Zhiyong Gao, “A Novel Deep Progressive Image Compression Framework,” in Picture Coding Symposium (PCS), pp. 1–5, Ningbo, China, November 2019.
- Yuan Tian, Xiongkuo Min, Guangtao Zhai, Zhiyong Gao, “Video-based early Autism Detection via Temporal Pyramid Networks,” in IEEE International Conference on Multimedia and Expo (ICME), 2019.
- Guo Lu, Wanli Ouyang, Dong Xu, Xiaoyun Zhang, Chunlei Cai, Zhiyong Gao, “DVC: An End-to-End Deep Video Compression Framework,” in IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
- Wenbo Bao, Chao Ma, Wei-Sheng Lai, Xiaoyun Zhang, Zhiyong Gao, Ming-Hsuan Yang, “Depth-Aware Video Frame Interpolation, ” in IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
- Shangpeng Yan, Wenbo Bao, Xiaoyun Zhang, Zhiyong Gao, Li Chen, “Large Scale Near-duplicate Image Retrieval via Patch Embedding”, in 4th International Workshop on Compact and Efficient Feature Representation and Learning in Computer Vision 2019.
- Shengyang Li, Xiaoyun Zhang, Xiaoxia Wang, Yumin Zhong, Xiaofen Yao, Ya Zhang,Yanfeng Wang. “Children’s Neuroblastoma Segmentation Using Morphological Features.” In International Workshop on Machine Learning in Medical Imaging, pp. 81-88. Springer, Cham, 2019.
- Lai, Bolin, Shiqi Peng, Guangyu Yao, Ya Zhang, Xiaoyun Zhang, Yanfeng Wang, and Hui Zhao. “Spatial Regularized Classification Network for Spinal Dislocation Diagnosis.” In International Workshop on Machine Learning in Medical Imaging, pp. 9-17. Springer, Cham, 2019.
- Peng, Shiqi, Bolin Lai, Guangyu Yao, Xiaoyun Zhang, Ya Zhang, Yan-Feng Wang, and Hui Zhao. “Learning-Based Bone Quality Classification Method for Spinal Metastasis.” In International Workshop on Machine Learning in Medical Imaging, pp. 426-434. Springer, Cham, 2019.
- Peng, Shiqi, Bolin Lai, Guangyu Yao, Xiaoyun Zhang, Ya Zhang, Yan-Feng Wang, and Hui Zhao. “Weakly Supervised Segmentation of Vertebral Bodies with Iterative Slice-Propagation.” In Domain Adaptation and Representation Transfer and Medical Image Learning with Less Labels and Imperfect Data, pp. 120-128. Springer, Cham, 2019.
- Yuan Tian, Zhaohui Che, Guangtao Zhai, Zhiyong Gao, “BAN, A Barcode Accurate Detection Network,” in IEEE International Conference on Visual Communications and Image Processing (VCIP), 2018.
- Cong Geng, Li Chen, Xiaoyun Zhang, Peng Zhou, Zhiyong Gao, “A Wavelet-based Learning for Face Hallucination with Loop Architecture,” in IEEE International Conference on Visual Communications and Image Processing (VCIP), 2018.
- Wenbo Bao, Xiaoyun Zhang, Li Chen, Zhiyong Gao, “KalmanFlow: Efficient Kalman Filtering for Video Optical Flow,” in IEEE International Conference on Image Processing (ICIP), 2018, pp. 3343-3347.
- Chunlei Cai, Li Chen, Lei Zhou, Xiaoyun Zhang, Zhiyong Gao, “Rcdfnn: Robust Change Detection Based on Convolutional Fusion Neural Network,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2018, pp. 1912-1916.
- Guo Lu, Wanli Ouyang, Dong Xu, Xiaoyun Zhang, Zhiyong Gao, Ming-ting Sun, “Deep Kalman Filtering Network for Video Compression Artifact Reduction, ” in European Conference on Computer Vision (ECCV), 2018, pp. 568-584.
- Chunmei Xie, Xiaoyun Zhang, Hua Yang, Li Chen, Zhiyong Gao, “Video Stitching Based on Optical Flow,” in IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), 2018.
- Lin Chen, Hua Yang, Shuang Wu, Zhiyong Gao, “Data generation for improving person re-identification,” in ACM on Multimedia Conference (MM), 2017, pp. 609-617.
- Lin Chen, Hua Yang, Ji Zhu, Qin Zhou, Shuang Wu, Zhiyong Gao, “Deep spatial-temporal fusion network for video-based person re-identification,” in IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2017, pp. 63-70.
- Wenbo Bao, Xiaoyun Zhang, Shangpeng Yan, Zhiyong Gao, “Iterative convolutional neural network for noisy image super-resolution,” in IEEE International Conference on Image Processing (ICIP), 2017, pp. 4038-4042.
- Guo Lu, Xiaoyun Zhang, Li Chen, Zhiyong Gao, “A Novel Frame Rate Up Conversion Using Iterative Non-Local Means Interpolation,” in IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), 2017.
- Wenbo Bao, Xiaoyun Zhang, Li Chen, Zhiyong Gao, “High-quality and real-time frame interpolation on heterogeneous computing system,” in IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), 2017.
- Chunlei Cai, Li Chen, Xiaoyun Zhang, Zhiyong Gao, Lonsn Liao, Jack Yu, “Moving segmentation in HEVC compressed domain based on logistic regression,” in IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), 2017.
Projects
- 国家自然科学基金(面上)，面向历史影像的深度先验驱动超分辨率， 2023.1-2026.12
- 上海市科委人工智能专项，基于深度生成模型的条件图像生成， 2022.9-2024.9
- 国家自然科学基金(面上)，帧率提升与压缩编码的联合优化， 2018.1-2021.12
- 国家自然科学基金(青年)，HEVC的低复杂度和并行编码方法， 2014.1-2016.12
- 国家重点研发计划课题， 4K超高清媒体的制作与分发关键技术，2020.1-2022.10
- 国家重点研发计划子课题，视频智能感知技术，2020.11-2023.10
- 核高基重大专项，高品质电视图像显示处理芯片研发及小批量应用，2013.1-2016.6
Openings
We are looking for Master or Ph.D candidates
- self-motivated with great enthusiastic and solid academic background
- be interested in deep learning and computer vision, especially in visual representation learning, generative AI models and its application in image and video processing, and the emerging AIGC
Please send your email to xiaoyun.zhang@sjtu.edu.cn
Contact
Cooperative Medianet Innovation Center
未来媒体网络协同创新中心
Shanghai Jiao Tong University
上海交通大学
Email: xiaoyun.zhang@sjtu.edu.cn
No. 5 EIEE Building East 303A
800 Dongchuan Rd. Minhang
Shanghai, 200240, China
TEL: +86-21-34205012
FAX: +86-21-34204155