Estimating Illumination Direction for Augmented Reality in Real-Time by using Low-Cost Sensors
Keywords:Augmented Reality, Illumination, Real- Time, Arduino, Android, Unity3d, Human Computer Interaction
In recent years, Augmented Reality has become a very popular topic, both as a research and commercial field. This trend has originated with the use of mobile devices as computational core and display. The appearance of virtual objects and their interaction with the real world is a key element in the success of an Augmented Reality software. A common issue in this type of software is the visual inconsistency between the virtual and real objects due to wrong illumination. Although illumination is a common research topic in Computer Graphics, few studies have been made about real time estimation of illumination direction. In this work we present a low-cost approach to detect the direction of the environment illumination, allowing the illumination of virtual objects according to the real light of the ambient, improving the integration of the scene. Our solution is open-source, based on Arduino hardware and the presented system was developed on Android.
 X. Zhang, S. Fronz, and N. Navab, “Visual marker detection and decoding in ar systems: A comparative study,” in Proceedings of the 1st International Symposium on Mixed and Augmented Reality, p. 97, IEEE Computer Society, 2002.
 D. Wagner, G. Reitmayr, A. Mulloni, T. Drummond, and D. Schmalstieg, “Real-time detection
and tracking for augmented reality on mobile phones,” IEEE transactions on visualization and computer graphics, vol. 16, no. 3, pp. 355–368, 2010.
 R. T. Azuma, “A survey of augmented reality,” Presence: Teleoperators and virtual environments, vol. 6, no. 4, pp. 355–385, 1997.
 D. Van Krevelen and R. Poelman, “A survey of augmented reality technologies, applications and limitations,” International Journal of Virtual Reality, vol. 9, no. 2, p. 1, 2010.
 P. Supan and I. Stuppacher, “Interactive image based lighting in augmented reality,” in Central European Seminar on Computer Graphics, vol. 17, p. 18, 2006.
 S. DiVerdi and T. Höllerer, “Combining dynamic physical and virtual illumination in augmented reality,” Environment, vol. 5, p. 12, 2004.
 K. Jacobs, J.-D. Nahmias, C. Angus, A. Reche, C. Loscos, and A. Steed, “Automatic generation of consistent shadows for augmented reality,” in Proceedings of Graphics Interface 2005, pp. 113–120, Canadian Human-Computer Communications Society, 2005.
 J.-F. Lalonde, A. A. Efros, and S. G. Narasimhan, “Estimating natural illumination from a single outdoor image,” in 2009 IEEE 12th International Conference on Computer Vision, pp. 183–190, IEEE, 2009.
 Y. Liu and X. Granier, “Online tracking of outdoor lighting variations for augmented reality with moving cameras,” IEEE Transactions on visualization and computer graphics, vol. 18, no. 4, pp. 573–580, 2012.
 M. Clements and A. Zakhor, “Interactive shadow analysis for camera heading in outdoor images,” in 2014 IEEE International Conference on Image Processing (ICIP), pp. 3367–3371, IEEE, 2014.
 J. Jachnik, R. A. Newcombe, and A. J. Davison, “Real-time surface light-ﬁeld capture for augmentation of planar specular surfaces,” in Mixed and Augmented Reality (ISMAR), 2012 IEEE International Symposium on, pp. 91–97, IEEE, 2012.
 G. Xing, X. Zhou, Q. Peng, Y. Liu, and X. Qin, “Lighting simulation of augmented outdoor scene
based on a legacy photograph,” in Computer Graphics Forum, vol. 32, pp. 101–110, Wiley Online Library, 2013.
 H. Kolivand, M. Kolivand, M. S. Sunar, and M. A. M. Arsad, “A fast silhouette detection algorithm for shadow volumes in augmented reality,” World Academy of Science, Engineering and Technology, International Journal of Computer, Electrical, Automation, Control and Information Engineering, vol. 10, no. 4, pp. 668–672, 2016.
 I. Arief, S. McCallum, and J. Y. Hardeberg, “Realtime estimation of illumination direction for augmented reality on mobile devices,” in Color and Imaging Conference, vol. 2012, pp. 111–116, Society for Imaging Science and Technology, 2012.
 K. Rohmer, W. Büschel, R. Dachselt, and T. Grosch, “Interactive near-ﬁeld illumination for photorealistic augmented reality on mobile devices,” in Mixed and Augmented Reality (ISMAR), 2014 IEEE International Symposium on, pp. 29–38, IEEE, 2014.
 L. Gruber, T. Langlotz, P. Sen, T. Höherer, and D. Schmalstieg, “Efﬁcient and robust radiance transfer for probeless photorealistic augmented reality,” in 2014 IEEE Virtual Reality (VR), pp. 15–20, IEEE, 2014.
 M. Slater, M. Usoh, and Y. Chrysanthou, “The inﬂuence of dynamic shadows on presence in immersive virtual environments,” in Virtual Environments 95, pp. 8–21, Springer, 1995.