Welcome to Ming Lin's Homepage


I am a Senior Applied Scientist at Amazon.com LCC. My research interests include Mathematical Foundation of Deep Learning and Statistical Machine Learning, with applications in deep learning acceleration, computer vision and mobile AI. Before I joined Amazon, I was a Staff Algorithm Engineer at DAMO Academy of Alibaba Group (U.S.) from April 2018 to July 2022. I was a Research Investigator in the Medical School of Michigan University from Sep 2015 to April 2018. I worked as a postdoctoral researcher in the School of Computer Science at Carnegie Mellon University from July 2014 to Sep 2015. I received my Ph.D. degree in computer science from Tsinghua University in 2014. During my Ph.D. study, I had been a visiting scholar in Michigan State University and in CMU from Dec 2013 to July 2014.


Email: minglamz@amazon.com or linming04@gmail.com

Office: 425 106th Ave NE, Bellevue, WA 98004. (Google Map)

[My LinkedIn] [My Google Scholar]


Public Talks

  • DeepMAD: Mathematical Architecture Design for Deep Learning. IFML Workshop 2023. [link]

  • Training-Free Approaches for Edge AI: Challenges, Opportunities and Progress. MLSys 2022. [link]

  • VALSE Talk [video] (Chinese)

Selected Papers

  • Xuan Shen, Yaohua Wang, Ming Lin, Yilun Huang, Hao Tang, Xiuyu Sun, Yanzhi Wang. DeepMAD: Mathematical Architecture Design for Deep Convolutional Neural Network. In Proceedings of the IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR), 2023. [arXiv]

  • Yiqi Jiang, Zhiyu Tan, Junyan Wang, Xiuyu Sun, Ming Lin, Hao Li. GiraffeDet: A Heavy-Neck Paradigm for Object Detection. In Proceedings of the International Conference on Learning Representations (ICLR), 2022. [paper] [GitHub] [bib]

  • Zhenzhong Lan, Ming Lin, Xuanchong Li, Alexander G. Hauptmann, Bhiksha Raj. Beyond Gaussian Pyramid: Multi-skip Feature Stacking for Action Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Page 204-212, 2015. [paper] [arXiv] [bib]


Projects

Tiny-NAS

Tiny-NAS is an integrated, distributed, full-stack framework for Zero-Shot Neural Architecture Search based on Pytorch and OpenMPI. It is able to design efficient deep neural networks for imagenet classification and object detectin within 1~2 hours on 8 GPUs. It also includes a hardware latency prediction module in order to optimize the network throughput for GPU or mobile phone.

  • Github repository is here.
  • Hugging-face online demo is here.

Zen-NAS

Zen-NAS is a lightning fast, training-free Neural Architecture Searching (NAS) algorithm for automatically designing deep neural networks with high prediction accuracy and high inference speed on GPU and mobile device. Using 1 GPU searching for 12 hours, ZenNAS is able to design networks of ImageNet top-1 accuracy comparable to EfficientNet-B5 (~83.6%) while inference speed 4.9x times faster on V100, 10x times faster on NVIDIA T4, 1.6x times faster on Google Pixel2.

GPU-Efficient Networks

This project aims to develop GPU-Efficient networks via automatic Neural Architecture Search techniques. This project is obsoleted as our ICCV 2021 work Zen-NAS is a more powerful tool for designing GPU-efficient networks.

  • Github repository (obsoleted) is here.


Publications

Conference

Xuan Shen, Yaohua Wang, Ming Lin, Yilun Huang, Hao Tang, Xiuyu Sun, Yanzhi Wang. DeepMAD: Mathematical Architecture Design for Deep Convolutional Neural Network. In Proceedings of the IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR), 2023. [arXiv] [Github]

Shuning Chang, Pichao Wang, Ming Lin, Fan Wang, David Junhao Zhang, Rong Jin, Mike Zheng Shou. Making Vision Transformers Efficient from A Token Sparsification View. In Proceedings of the IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR), 2023. [arXiv]

Junyan Wang, Zhenhong Sun, Yichen Qian, Dong Gong, Xiuyu Sun, Ming Lin, Maurice Pagnucco, Yang Song. Maximizing Spatio-Temporal Entropy of Deep 3D CNNs for Efficient Video Recognition. In Proceedings of the Eleventh International Conference on Learning Representations (ICLR), 2023. [OpenReview]

Yaohua Wang, Fangyi Zhang, Ming Lin, Senzhang Wang, Xiuyu Sun, Rong Jin. Robust Graph Structure Learning over Images via Multiple Statistical Tests. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS), 2022. [OpenReview] [bib]

Zhenhong Sun, Ce Ge, Junyan Wang, Ming Lin, Hesen Chen, Hao Li, Xiuyu Sun. Entropy-Driven Mixed-Precision Quantization for Deep Network Design on IoT Devices. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS), 2022.[OpenReview] [bib]

Pichao Wang, Xue Wang, Fan Wang, Ming Lin, Shuning Chang, Hao Li, Rong Jin. KVT: k-NN Attention for Boosting Vision Transformers. In Proceedings of the European Conference on Computer Vision (ECCV), 2022. [paper] [bib]

Yaohua Wang, Fangyi Zhang, Ming Lin, Senzhang Wang, Xiuyu Sun, Rong Jin. Robust Graph Structure Learning over Images via Multiple Statistical Tests. In Proceedings of the Conference on Neural Information Processing Systems (NIPS), 2022. [paper]

Zhenhong Sun, Ming Lin, Xiuyu Sun, Zhiyu Tan, Hao Li and Rong Jin. MAE-DET: Revisiting Maximum Entropy Principle in Zero-Shot NAS for Efficient Object Detection. In Proceedings of the International Conference on Machine Learning (ICML), 2022. [paper] [arxiv] [GitHub] [bib]

Yichen Qian, Xiuyu Sun, Ming Lin, Zhiyu Tan, Rong Jin. Entroformer: A Transformer-based Entropy Model for Learned Image Compression. In Proceedings of the International Conference on Learning Representations (ICLR), 2022. [paper] [GitHub] [bib]

Yaohua Wang, Yaobin Zhang, Fangyi Zhang, Senzhang Wang, Ming Lin, YuQi Zhang, Xiuyu Sun. Ada-NETS: Face Clustering via Adaptive Neighbour Discovery in the Structure Space. In Proceedings of the International Conference on Learning Representations (ICLR), 2022. [paper] [GitHub] [bib]

Yiqi Jiang, Zhiyu Tan, Junyan Wang, Xiuyu Sun, Ming Lin, Hao Li. GiraffeDet: A Heavy-Neck Paradigm for Object Detection. In Proceedings of the International Conference on Learning Representations (ICLR), 2022. [paper] [GitHub] [bib]

Ming Lin, Pichao Wang, Zhenhong Sun, Hesen Chen, Xiuyu Sun, Qi Qian, Hao Li, Rong Jin. Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021. [paper] [arXiv] [bib]

Yichen Qian, Zhiyu Tan, Xiuyu Sun, Ming Lin, Dongyang Li, Zhenhong Sun, Li Hao, Rong Jin. Learning Accurate Entropy Model with Global Reference for Image Compression. In Proceedings of the International Conference on Learning Representations (ICLR), 2021. [paper] [bib]

Ming Lin, Xiaomin Song, Qi Qian, Hao Li, Liang Sun, Shenghuo Zhu, Rong Jin. Robust Gaussian Process Regression for Real-Time High Precision GPS Signal Enhancement. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (SIGKDD), 2019. [paper] [arXiv] [bib]

Hesen Chen, Ming Lin, Xiuyu Sun, Qian Qi, Hao Li, Rong Jin. MuffNet: Multi-Layer Feature Federation for Mobile Deep Learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV workshop), 2019. [paper] [bib]

Ming Lin, Shuang Qiu, Jieping Ye, Xiaomin Song, Qi Qian, Liang Sun, Shenghuo Zhu, Rong Jin. Which Factorization Machine Modeling is Better: A Theoretical Answer with Optimal Guarantee. In Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence (AAAI), 2019. [paper] [arXiv] [bib]

Tieliang Gong, Guangtao Wang, Jieping Ye, Zongben Xu, Ming Lin. Margin Based PU Learning. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence (AAAI), 2018. [paper] [appendix] [bib]

Xiang Li, Aoxiao Zhong, Ming Lin, Ning Guo, Mu Sun, Arkadiusz Sitek, Jieping Ye, James Thrall, Quanzheng Li. Self-paced Convolutional Neural Network for Computer Aided Detection in Medical Imaging Analysis. In Proceedings of the 8th International Workshop on Machine Learning in Medical Imaging (MLMI), 2017. [paper] [arXiv] [bib]

Zhenzhong Lan, Shoou-I Yu, Dezhong Yao, Ming Lin, Bhiksha Raj ; Alexander Hauptmann. The Best of BothWorlds: Combining Data-Independent and Data-Driven Approaches for Action Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Pages 1196-1205, 2016. [paper] [arXiv] [bib] [Github]

Ming Lin, Jieping Ye. A Non-convex One-Pass Framework for Generalized Factorization Machine and Rank-One Matrix Sensing. In Proceedings of the 30th Annual Conference on Neural Information Processing Systems (NIPS), Pages 1633-1641, 2016. [paper] [arXiv] [bib] [Github]

Chuang Gang, Ming Lin, Yi Yang, Gerard de Melo, Alexander G. Hauptmann. Concepts Not Alone: Exploring Pairwise Relationships for Zero-Shot Video Activity Recognition. In Proceedings of the 30th AAAI Conference on Artificial Intelligence (AAAI), Pages 3487-3493, 2016. [paper] [bib]

Chuang Gang, Ming Lin, Yi Yang, Alexander G. Hauptmann. Exploring Semantic Inter-Class Relationships (SIR) for Zero-Shot Action Recognition. In Proceedings of the the 29th AAAI Conference on Artificial Intelligence (AAAI), Pages 3769-3775, 2015. [paper] [bib]

Zhenzhong Lan, Ming Lin, Xuanchong Li, Alexander G. Hauptmann, Bhiksha Raj. Beyond Gaussian Pyramid: Multi-skip Feature Stacking for Action Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Page 204-212, 2015. [paper] [arXiv] [bib]

Ming Lin, Zhenzhong Lan, Alexander G. Hauptmann. Density Corrected Sparse Recovery when R.I.P. Condition is Broken. In Proceedings of the 24th International Joint Conference on Artificial Intelligence (IJCAI), Pages 3664-3670, 2015. [paper] [bib]

Shoou-I Yu, Lu Jiang, Zexi Mao, Xiaojun Chang, Xingzhong Du, Chuang Gan, Zhenzhong Lan, Zhongwen Xu, Xuanchong Li, Yang Cai, Anurag Kumar, Yajie Miao, Lara Martin, Nikolas Wolfe, Shicheng Xu, Huan Li, Ming Lin, Zhigang Ma, Yi Yang, Deyu Meng, Shiguang Shan, Pinar Duygulu Sahin, Susanne Burger, Florian Metze, Rita Singh, Bhiksha Raj, Teruko Mitamura, Richard Stern, Alexander Hauptmann. Informedia@ trecvid 2014 med and mer. NIST TRECVID Video Retrieval Evaluation Workshop, 2014. [paper] [bib]

Ming Lin, Rong Jin, Changshui Zhang. Efficient Sparse Recovery via Adaptive Non-Convex Regularizers with Oracle Property. In Proceedings of the 30th Conference on Uncertainty in Artificial Intelligenre (UAI), Pages 505-514, 2014. [paper] [acm] [appendix] [bib]

Lijun Zhang, Jinfeng Yi, Ming Lin, Xiaofei He. Online Kernel Learning with a Near Optimal Sparsity Bound. In Proceedings of the 30th International Conference on Machine Learning (ICML), pages 621 – 629, 2013. [paper] [bib]

Journal

Jian Liang, Kun Chen, Ming Lin, Changshui Zhang, Fei Wang. Robust Finite Mixture Regression for Heterogeneous Targets. Data Mining and Knowledge Discovery, Volume 32, Issue 6, pp 1509–1560, November 2018. [paper] [arXiv] [bib]

Ming Lin, Pinghua Gong, Tao Yang, Jieping Ye, Roger L. Albin, Hiroko H. Dodge. Big Data Analytical Approaches to the NACC Dataset: Aiding Preclinical Trial Enrichment. Alzheimer Dis Assoc Disord. Volume 32, Issue 1, Pages 18-27, 2018. [paper] [bib]

Daqing Chang, Ming Lin, Changshui Zhang. On the generalization ability of online gradient descent algorithm under the quadratic growth condition. IEEE Transactions on Neural Networks and Learning Systems, vol. 29, no. 10, pp. 5008-5019, October 2018. [paper] [bib]

Ming Lin, Vaibhav Narayan, Wayne C. Drevets, Jieping Ye, Qingqin Li. Application of Growth Mixture Modeling in Antidepressant Treatment Response Studies. Biological Psychiatry, Volume 81, Issue 10, Supplement, Page S224, May 2017. [paper] [ris]

Xiaojun Chang, Zhigang Ma, Ming Lin, Yi Yang, Alexander G. Hauptmann. Feature Interaction Augmented Sparse Learning for Fast Kinect Motion Detection. IEEE Transactions on Image Processing, Volume 26, Issue 8, Pages 3911-3920. 2017. [paper] [bib]

Shoou-I Yu, Yi Yang, Zhongwen Xu, Shicheng Xu, Deyu Meng, Zexi Mao, Zhigang Ma, Ming Lin, Xuanchong Li, Huan Li, Zhenzhong Lan, Lu Jiang, Alexander G. Hauptmann, Chuang Gan, Xingzhong Du, Xiaojun Chang. Strategies for Searching Video Content with Text Queries or Video Examples (Invited Paper). ITE Transactions on Media Technology and Applications 4.3, Pages 227-238, 2016. [paper] [arXiv] [bib]

Ming Lin, Lijun Zhang, Rong Jin, Shifeng Weng, Changshui Zhang. Online Kernel Learning with Nearly Constant Support Vectors. Neurocomputing. Volume 179, Pages 26–36, 2016. [paper] [bib]

Zheng Hu, Ming Lin, Changshui Zhang. Dependent Online Kernel Learning with Constant Number of Random Fourier Features. IEEE Transactions on Neural Networks and Learning Systems (TNNLS), Volume 26, Issue 10, Pages 2464-2476, 2015. [paper] [bib]

Zheng Pan, Ming Lin, Guangdong Hou, Changshui Zhang. Damping proximal coordinate descent algorithm for non-convex regularization. Neurocomputing, vol 152 Pages 151-163, 2015. [paper] [bib]

Ming Lin, Fei Wang, Changshui Zhang. Large-Scale Eigenvector Approximation via Hilbert Space Embedding Nystrom. Pattern Recognition (PR), 48(5), Pages 1904-1912, 2015. [paper] [bib]

Ming Lin, Shifeng Weng, Changshui Zhang. On the Sample Complexity of Random Fourier Features for Online Learning: How Many Random Fourier Features Do We Need ?. ACM Transactions on Knowledge Discovery from Data (TKDD), Volume 8 Issue 3, Pages 13:1--13:19, June 2014, ISSN 1556-4681. [paper] [bib]

Shizhun Yang, Ming Lin, Chenping Hou, Changshui Zhang, Yi Wu. A General Framework for Transfer Sparse Subspace Learning. Neural Computing and Applications. Volume 21, Number 7, Pages 1801-1817, 2012. [paper] [bib]

Pre-print

Pichao Wang, Xue Wang, Fan Wang, Ming Lin, Shuning Chang, Wen Xie, Hao Li, Rong Jin. KVT: k-NN Attention for Boosting Vision Transformers. 2021. [arXiv]

Yonathan Aflalo, Asaf Noy, Ming Lin, Itamar Friedman, Lihi Zelnik. Knapsack Pruning with Inner Distillation. 2020. [arXiv]

Yi Xu, Asaf Noy, Ming Lin, Qi Qian, Hao Li, Rong Jin. WeMix: How to Better Utilize Data Augmentation. 2020. [arXiv]

Ming Lin, Hesen Chen, Xiuyu Sun, Qi Qian, Hao Li, Rong Jin. Neural Architecture Design for GPU-Efficient Networks. 2020. [arXiv] [Github]

Zhenzhong Lan, Xuanchong Li, Ming Lin, Alexander G. Hauptmann. Long-short term motion feature for action classification and retrieval. 2015. [arXiv]

Zhenzhong Lan, Shoou-I Yu, Ming Lin, Bhiksha Raj, Alexander G. Hauptmann. Handcrafted local features are convolutional neural networks. 2015. [arXiv]

Shuang Qiu, Tingjin Luo, Jieping Ye, Ming Lin. Nonconvex One-bit Single-label Multi-label Learning. 2017. [arXiv]

Ming Lin, Shuang Qiu, Bin Hong, Jieping Ye. The Second Order Linear Model. 2017. [arXiv] [Github]