A Mutual Learning Framework for Pruned and Quantized Networks

Xiaohai Li; Yiqiang Chen; Jindong Wang

doi:10.24215/16666038.23.e01

Authors

Xiaohai Li Institute of Computing Technology, Chinese Academy of Sciences
Yiqiang Chen Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China
Jindong Wang Microsoft Research Asia, Beijing, China

DOI:

https://doi.org/10.24215/16666038.23.e01

Keywords:

model compression, network pruning, quantization, mutual learning

Abstract

Model compression is an important topic in deep learning research. It can be mainly divided into two directions: model pruning and model quantization. However, both methods will more or less affect the original accuracy of the model. In this paper, we propose a mutual learning framework for pruned and quantized networks. We regard the pruned network and the quantizated network as two sets of features that are not parallel. The purpose of our mutual learning framework is to better integrate the two sets of features and achieve complementary advantages, which we call it feature augmentation. To verify the effectiveness of our framework, we select a pairwise combination of 3 state-of-the-art pruning algorithms and 3 state-of-theart quantization algorithms. Extensive experiments on CIFAR-10, CIFAR-100 and Tiny-imagenet show the benefits of our framework: through the mutual learning of the two networks, we obtain a pruning network and a quantization network with higher accuracy at the same time.

Downloads

Download data is not yet available.

References

A. Ankit, T. Ibrayev, A. Sengupta, and K. Roy, “Trannsformer: Clustered pruning on crossbar-based architectures for energy-efficient neural networks,” IEEE Trans. Comput. Aided Des. Integr. Circuits Syst., vol. 39, no. 10, pp. 2361–2374, 2020.

S. Wang, H. Cai, J. A. Bilmes, and W. S. Noble, “Training compressed fully-connected networks with a density-diversity penalty,” in 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings, OpenReview.net, 2017.

F. Meng, H. Cheng, K. Li, H. Luo, X. Guo, G. Lu, and X. Sun, “Pruning filter in filter,” in Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual (H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, eds.), 2020.

W. Wang, C. Fu, J. Guo, D. Cai, and X. He, “COP: customized deep model compression via regularized correlation-based filter-level pruning,” CoRR, vol. abs/1906.10337, 2019.

Y. Woo, D. Kim, J. Jeong, Y. W. Ko, and J. Lee, “Zerokeep filter pruning for energy efficient deep neural network,” in International Conference on Information and Communication Technology Convergence, ICTC 2020, Jeju Island, Korea (South), October 21-23, 2020, pp. 1288–1292, IEEE, 2020.

C. Zhao, B. Ni, J. Zhang, Q. Zhao, W. Zhang, and Q. Tian, “Variational convolutional neural network pruning,” in IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pp. 2780–2789, Computer Vision Foundation / IEEE, 2019.

P. Molchanov, S. Tyree, T. Karras, T. Aila, and J. Kautz, “Pruning convolutional neural networks for resource efficient inference,” in 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings, OpenReview.net, 2017.

B. Dai, C. Zhu, B. Guo, and D. P. Wipf, “Compressing neural networks using the variational information bottleneck,” in Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsm ¨assan, Stockholm, Sweden, July 10-15, 2018 (J. G. Dy and A. Krause, eds.), vol. 80 of Proceedings of Machine Learning Research, pp. 1143–1152, PMLR, 2018.

S. Han, J. Pool, J. Tran, andW. J. Dally, “Learning both weights and connections for efficient neural network,” in Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada (C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, eds.), pp. 1135–1143, 2015.

W. Wen, C. Wu, Y. Wang, Y. Chen, and H. Li, “Learning structured sparsity in deep neural networks,” in Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain (D. D. Lee, M. Sugiyama, U. von Luxburg, I. Guyon, and R. Garnett, eds.), pp. 2074–2082, 2016.

Y. Guo, A. Yao, and Y. Chen, “Dynamic network surgery for efficient dnns,” in Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain (D. D. Lee, M. Sugiyama, U. von Luxburg, I. Guyon, and R. Garnett, eds.), pp. 1379–1387, 2016.

X. Ding, G. Ding, X. Zhou, Y. Guo, J. Han, and J. Liu, “Global sparse momentum SGD for pruning very deep neural networks,” in Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada (H. M. Wallach, H. Larochelle, A. Beygelzimer, F. d’Alch´e- Buc, E. B. Fox, and R. Garnett, eds.), pp. 6379–6391, 2019.

V. Sanh, T. Wolf, and A. M. Rush, “Movement pruning: Adaptive sparsity by fine-tuning,” in Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual (H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, eds.), 2020.

Y. Ro and J. Y. Choi, “Autolr: Layer-wise pruning and auto-tuning of learning rates in fine-tuning of deep networks,” in Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021, pp. 2486–2494, AAAI Press, 2021.

H. Li, A. Kadav, I. Durdanovic, H. Samet, and H. P. Graf, “Pruning filters for efficient convnets,” in 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings, OpenReview.net, 2017.

J. Luo, J. Wu, and W. Lin, “Thinet: A filter level pruning method for deep neural network compression,” in IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, October 22-29, 2017, pp. 5068–5076, IEEE Computer Society, 2017.

Y. He, X. Zhang, and J. Sun, “Channel pruning for accelerating very deep neural networks,” in IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, October 22-29, 2017, pp. 1398–1406, IEEE Computer Society, 2017.

M. Lin, R. Ji, Y. Wang, Y. Zhang, B. Zhang, Y. Tian, and L. Shao, “Hrank: Filter pruning using high-rank feature map,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pp. 1526–1535, Computer Vision Foundation / IEEE, 2020.

J. Chen, Z. Zhu, C. Li, and Y. Zhao, “Self-adaptive network pruning,” in Neural Information Processing - 26th International Conference, ICONIP 2019, Sydney, NSW, Australia, December 12-15, 2019, Proceedings, Part I (T. Gedeon, K. W. Wong, and M. Lee, eds.), vol. 11953 of Lecture Notes in Computer Science, pp. 175–186, Springer, 2019.

Y. Wang, X. Zhang, X. Hu, B. Zhang, and H. Su, “Dynamic network pruning with interpretable layerwise channel selection,” in The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020, pp. 6299–6306, AAAI Press, 2020.

S. Chao, Z. Wang, Y. Xing, and G. Cheng, “Directional pruning of deep neural networks,” in Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual (H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, eds.), 2020.

I. Jung, K. You, H. Noh, M. Cho, and B. Han, “Realtime object tracking via meta-learning: Efficient model adaptation and one-shot channel pruning,” in The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020, pp. 11205–11212, AAAI Press, 2020.

M. S. Zhang and B. C. Stadie, “One-shot pruning of recurrent neural networks by jacobian spectrum evaluation,” in 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020, OpenReview.net, 2020.

H. Tanaka, D. Kunin, D. L. K. Yamins, and S. Ganguli, “Pruning neural networks without any data by iteratively conserving synaptic flow,” in Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual (H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, eds.), 2020.

T. Jiang, X. Yang, Y. Shi, and H. Wang, “Layer-wise deep neural network pruning via iteratively reweighted optimization,” in IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2019, Brighton, United Kingdom, May 12-17, 2019, pp. 5606–5610, IEEE, 2019.

M. Courbariaux, Y. Bengio, and J. David, “Binaryconnect:Training deep neural networks with binary weights during propagations,” in Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada (C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, eds.), pp. 3123–3131, 2015.

S. Darabi, M. Belbahri, M. Courbariaux, and V. P. Nia, “BNN+: improved binary network training,” CoRR, vol. abs/1812.11800, 2018.

H. Phan, D. Huynh, Y. He, M. Savvides, and Z. Shen, “Mobinet: A mobile binary network for image classification,” in IEEE Winter Conference on Applications of Computer Vision, WACV 2020, Snowmass Village, CO, USA, March 1-5, 2020, pp. 3442–3451, IEEE, 2020.

A. Shekhovtsov, V. Yanush, and B. Flach, “Path sample-analytic gradient estimators for stochastic binary networks,” in Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual (H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, eds.), 2020.

D. Kim, K. P. Singh, and J. Choi, “Learning architectures for binary networks,” in Computer Vision - ECCV 2020 - 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XII (A. Vedaldi, H. Bischof, T. Brox, and J. Frahm, eds.), vol. 12357 of Lecture Notes in Computer Science, pp. 575–591, Springer, 2020.

A. Bulat, B. Mart´ınez, and G. Tzimiropoulos, “Highcapacity expert binary networks,” in 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021, OpenReview.net, 2021.

K. Wang, Z. Liu, Y. Lin, J. Lin, and S. Han, “HAQ: hardware-aware automated quantization with mixed precision,” in IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pp. 8612–8620, Computer Vision Foundation / IEEE, 2019.

S. Uhlich, L. Mauch, F. Cardinaux, K. Yoshiyama, J. A. Garc´ıa, S. Tiedemann, T. Kemp, and A. Nakamura, “Mixed precision dnns: All you need is a good parametrization,” in 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020, OpenReview.net, 2020.

X. Liu, M. Ye, D. Zhou, and Q. Liu, “Post-training quantization with multiple points: Mixed precision without mixed precision,” in Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021, pp. 8697– 8705, AAAI Press, 2021.

L. Yang and Q. Jin, “Fracbits: Mixed precision quantization via fractional bit-widths,” in Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021, pp. 10612–10620, AAAI Press, 2021.

H. Yang, L. Duan, Y. Chen, and H. Li, “BSQ: exploring bit-level sparsity for mixed-precision neural network quantization,” in 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021, OpenReview.net, 2021.

A. Goncharenko, A. Denisov, S. Alyamkin, and E. Terentev, “On practical approach to uniform quantization of non-redundant neural networks,” in Artificial Neural Networks and Machine Learning - ICANN 2019: Deep Learning - 28th International Conference on Artificial Neural Networks, Munich, Germany, September 17-19, 2019, Proceedings, Part II (I. V. Tetko, V. Kurkov´a, P. Karpov, and F. J. Theis, eds.), vol. 11728 of Lecture Notes in Computer Science, pp. 349–360, Springer, 2019.

Z. Cai, X. He, J. Sun, and N. Vasconcelos, “Deep learning with low precision by half-wave gaussian quantization,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017, pp. 5406–5414, IEEE Computer Society, 2017.

J. Faraone, N. J. Fraser, M. Blott, and P. H. W. Leong, “SYQ: learning symmetric quantization for efficient deep neural networks,” in 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pp. 4300– 4309, Computer Vision Foundation / IEEE Computer Society, 2018.

S. Gupta, A. Agrawal, K. Gopalakrishnan, and P. Narayanan, “Deep learning with limited numerical precision,” in Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015 (F. R. Bach and D. M. Blei, eds.), vol. 37 of JMLR Workshop and Conference Proceedings, pp. 1737–1746, JMLR.org, 2015.

S. Jung, C. Son, S. Lee, J. Son, J. Han, Y. Kwak, S. J. Hwang, and C. Choi, “Learning to quantize deep networks by optimizing quantization intervals with task loss,” in IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pp. 4350–4359, Computer Vision Foundation / IEEE, 2019.

Z. Liao, R. Couillet, and M. W. Mahoney, “Sparse quantized spectral clustering,” in 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021, OpenReview.net, 2021.

E. Park, J. Ahn, and S. Yoo, “Weighted-entropy-based quantization for deep neural networks,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017, pp. 7197–7205, IEEE Computer Society, 2017.

E. Park, S. Yoo, and P. Vajda, “Value-aware quantization for training and inference of neural networks,” in Computer Vision - ECCV 2018 - 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part IV (V. Ferrari, M. Hebert, C. Sminchisescu, and Y. Weiss, eds.), vol. 11208 of Lecture Notes in Computer Science, pp. 608–624, Springer, 2018.

P. Gysel, J. J. Pimentel, M. Motamedi, and S. Ghiasi, “Ristretto: A framework for empirical study of resourceefficient inference in convolutional neural networks,” IEEE Trans. Neural Networks Learn. Syst., vol. 29, no. 11, pp. 5784–5789, 2018.

I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and Y. Bengio, “Binarized neural networks,” in Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain (D. D. Lee, M. Sugiyama, U. von Luxburg, I. Guyon, and R. Garnett, eds.), pp. 4107–4115, 2016.

M. Rastegari, V. Ordonez, J. Redmon, and A. Farhadi, “Xnor-net: Imagenet classification using binary convolutional neural networks,” in Computer Vision - ECCV 2016 - 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part IV (B. Leibe, J. Matas, N. Sebe, and M. Welling, eds.), vol. 9908 of Lecture Notes in Computer Science, pp. 525–542, Springer, 2016.

S. A. Tailor, J. Fern´andez-Marqu´es, and N. D. Lane, “Degree-quant: Quantization-aware training for graph neural networks,” in 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021, OpenReview.net, 2021.

Y. Cai, Z. Yao, Z. Dong, A. Gholami, M. W. Mahoney, and K. Keutzer, “Zeroq: A novel zero shot quantization framework,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pp. 13166–13175, Computer Vision Foundation / IEEE, 2020.

Y. Choukroun, E. Kravchik, F. Yang, and P. Kisilev, “Low-bit quantization of neural networks for efficient inference,” in 2019 IEEE/CVF International Conference on Computer Vision Workshops, ICCV Workshops 2019, Seoul, Korea (South), October 27-28, 2019, pp. 3009–3018, IEEE, 2019.

X. He and J. Cheng, “Learning compression from limited unlabeled data,” in Computer Vision - ECCV 2018 - 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part I (V. Ferrari, M. Hebert, C. Sminchisescu, and Y. Weiss, eds.), vol. 11205 of Lecture Notes in Computer Science, pp. 778–795, Springer, 2018.

E. Meller, A. Finkelstein, U. Almog, and M. Grobman, “Same, same but different: Recovering neural network quantization error through weight factorization,” in Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA (K. Chaudhuri and R. Salakhutdinov, eds.), vol. 97 of Proceedings of Machine Learning Research, pp. 4486–4495, PMLR, 2019.

M. Nagel, M. van Baalen, T. Blankevoort, and M. Welling, “Data-free quantization through weight equalization and bias correction,” in 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019, pp. 1325–1334, IEEE, 2019.

R. Zhao, Y. Hu, J. Dotzel, C. D. Sa, and Z. Zhang, “Improving

neural network quantization without retraining using outlier channel splitting,” in Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA (K. Chaudhuri and R. Salakhutdinov, eds.), vol. 97 of Proceedings of Machine Learning Research, pp. 7543–7552, PMLR, 2019.

F. Tung and G. Mori, “CLIP-Q: deep network compression learning by in-parallel pruning-quantization,” in 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pp. 7873–7882, Computer Vision Foundation / IEEE Computer Society, 2018.

T. Wang, K. Wang, H. Cai, J. Lin, Z. Liu, H. Wang, Y. Lin, and S. Han, “APQ: joint search for network architecture, pruning and quantization policy,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pp. 2075–2084, Computer Vision Foundation / IEEE, 2020.

Y. Wang, Y. Lu, and T. Blankevoort, “Differentiable joint pruning and quantization for hardware efficiency,” in Computer Vision - ECCV 2020 - 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXIX (A. Vedaldi, H. Bischof, T. Brox, and J. Frahm, eds.), vol. 12374 of Lecture Notes in Computer Science, pp. 259–277, Springer, 2020.

M. van Baalen, C. Louizos, M. Nagel, R. A. Amjad, Y. Wang, T. Blankevoort, and M. Welling, “Bayesian bits: Unifying quantization and pruning,” in Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual (H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, eds.), 2020.

J. Kim, K. Yoo, and N. Kwak, “Position-based scaled gradient for model quantization and pruning,” in Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual (H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, eds.), 2020.

S. Chao, Z. Wang, Y. Xing, and G. Cheng, “Directional pruning of deep neural networks,” in Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, eds.), 2020.

Y. Li, S. Lin, J. Liu, Q. Ye, M. Wang, F. Chao, F. Yang, J. Ma, Q. Tian, and R. Ji, “Towards compact cnns via collaborative compression,” in IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021, pp. 6438–6447, Computer Vision Foundation / IEEE, 2021.

H. Wang, C. Qin, Y. Zhang, and Y. Fu, “Neural pruning via growing regularization,” in 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021, OpenReview.net, 2021.

Z. Xu, M. Lin, J. Liu, J. Chen, L. Shao, Y. Gao, Y. Tian, and R. Ji, “Recu: Reviving the dead weights in binary neural networks,” in 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021, pp. 5178–5188, IEEE, 2021.

H. Yu, H. Li, H. Shi, T. S. Huang, and G. Hua, “Any-precision deep neural networks,” in Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021, pp. 10763–10771, AAAI Press, 2021.

J. Lee, D. Kim, and B. Ham, “Network quantization with element-wise gradient scaling,” in IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021, pp. 6448–6457, Computer Vision Foundation / IEEE, 2021.