Neural Network Techniques for Time Series Prediction: A Review

Muhammad Mushtaq - Universiti Tun Hussein Onn Malaysia, Johor, Malaysia
Urooj Akram - Universiti Tun Hussein Onn Malaysia, Johor, Malaysia
Muhammad Aamir - Universiti Tun Hussein Onn Malaysia, Johor, Malaysia
Haseeb Ali - Universiti Tun Hussein Onn Malaysia, Johor, Malaysia
Muhammad Zulqarnain - Universiti Tun Hussein Onn Malaysia, Johor, Malaysia


Citation Format:



DOI: http://dx.doi.org/10.30630/joiv.3.3.281

Abstract


It is important to predict a time series because many problems that are related to prediction such as health prediction problem, climate change prediction problem and weather prediction problem include a time component. To solve the time series prediction problem various techniques have been developed over many years to enhance the accuracy of forecasting. This paper presents a review of the prediction of physical time series applications using the neural network models. Neural Networks (NN) have appeared as an effective tool for forecasting of time series.  Moreover, to resolve the problems related to time series data, there is a need of network with single layer trainable weights that is Higher Order Neural Network (HONN) which can perform nonlinearity mapping of input-output. So, the developers are focusing on HONN that has been recently considered to develop the input representation spaces broadly. The HONN model has the ability of functional mapping which determined through some time series problems and it shows the more benefits as compared to conventional Artificial Neural Networks (ANN). The goal of this research is to present the reader awareness about HONN for physical time series prediction, to highlight some benefits and challenges using HONN.

Keywords


Time series prediction, Neural Network, Forecasting, Higher Order Neural Network, Physical time series.

Full Text:

PDF

References


Hussain, A. J., Knowles, A., Lisboa, P. J. G. & El-Deredy, W., “Financial time series prediction using polynomial pipelined neural networks,†Expert Systems with Applications., vol. 35, no. 3, pp. 1186–1199, 2008.

Ghazali, R., Jaafar Hussain, A., Mohd Nawi, N. & Mohamad, B, “Non-stationary and stationary prediction of financial time series using dynamic ridge polynomial neural network,†Neurocomputing, vol. 72, no. 10–12, pp. 2359–2367, 2009.

Sewell, M. V, “The Application of Intelligent Systems to Financial Time Series Analysis,†PhD thesis, Department of Computer Science, University College London, 2012.

Adhikari, R. & Agrawal, R. K., “A combination of artificial neural network and random walk models for financial time series forecasting,†Neural Comput. Appl., vol. 24, no. 6, pp. 1441–1449, 2013.

Mahdi, A. A., Hussain, A. J. & Al-Jumeily, D, “The prediction of non-stationary physical time series using the application of regularization technique in self-organised multilayer perceptrons inspired by the immune algorithm,†Proc. of the 3rd International Conference on Developments in E-Systems Engineering., pp. 213–218, 2010.

Malik, N., “Artificial Neural Networks and their applications,†Neural and Evolutionary Computing., 2005.

Mushtaq, M. F., Akram, U., Tariq, A., Khan, I., Zulqarnain, M., & Iqbal, U., An Innovative Cognitive Architecture for Humanoid Robot. International Journal of Advanced Computer Science and Applications., vol. 8, no. 8, pp. 60–67, 2017.

Stergiou, C. & Siganos, D, Neural Networks, vol. 47. 2016.

Mushtaq, M. F., Khan, D. M., Akram, U., Ullah, S., & Tariq, A., A Cognitive Architecture for Self Learning in Humanoid Robots, International Journal of Computer Science and Network Security., vol. 17, no. 5, pp. 26–36, 2017.

Stiles, J. & Jernigan, T. L., “The Basics of Brain Development,†Neuropsychol Rev, pp. 327–348, 2010.

M. Zulqarnain, R. Ghazali, Y. Mazwin, A. K. Z. Al Saedi, and U. Akram, “A comprehensive review on text classification using deep learning,†in Journal of Physics: Conference Series, 2018.

Shen, J., Su, P., Cheung, S. S., Member, S. & Zhao, J, “Virtual Mirror Rendering With Stationary RGB-D Cameras and Stored 3-D Background,†I IEEE Transactions on Image Processing., vol. 22, no. 9, pp. 3433–3448, 2013.

Zhang, G. P., Patuwo, B. E. & Hu, M. Y. A simulation study of artificial neural networks for nonlinear time-series forecasting. Computers & Operations Research., vol. 28, pp. 381–396, 2001.

Nayak, J., Kanungo, D. P., Naik, B. & Behera, H. S. A higher order evolutionary Jordan pi-sigma neural network with gradient descent learning for classification. In International Conference on High Performance Computing and Applications, pp. 1–6, 2015.

Hassim, Y. M. M. & Ghazali, R. Using Artificial Bee Colony to Improve Functional Link Neural Network Training. Applied Mechanics and Materials, 266., vol. 266, pp. 2102–2108, 2013.

Hussain, A. J., Al-Jumeily, D., Al-Askar, H. & Radi, N, “Regularized dynamic self-organized neural network inspired by the immune algorithm for financial time series prediction,†Neurocomputing, vol. 188, pp. 23–30, 2016.

Pagariya, R. & Bartere, M. “Review Paper on Artificial Neural Networksâ€. International Journal of Advanced Research in Computer Science, 4(6), pp. 49–53, 2013.

Moustra, M., Avraamides, M. & Christodoulou, C. “Expert Systems with Applications Artificial neural networks for earthquake prediction using time series magnitude data or Seismic Electric Signalsâ€. Expert Systems with Applications, 38(12), pp. 15032–15039, 2011.

Zamani, A. & Sorbi, M. R. (2013). “Application of neural network and ANFIS model for earthquake occurrence in Iranâ€. Earth Science Informatics 6(2), pp. 71–85, 2013.

C. Serrano-Cinca, “Feedforward neural networks in the classification of financial information,†The European Journal of Finance., vol. 3, pp. 183–202, 1997.

Sharma, P., Malik, N., Akhtar, N., Rahul & Rohilla, H. Feedforward Neural Network: A Review. International Journal of Advanced Research in Engineering and Applied Sciences., vol. 2, no. 10, pp. 25–34, 2013.

Abdulkarim, S. A. & Garko, A. B. (2015). Evaluating Feedforward and Elman Recurrent Neural Network Performances in Time Series Forecasting. Dutse Journal of Pure and Applied Sciences., vol. 1, no. June, pp. 145–151, 2015.

Wu, H., Zhou, Y., Luo, Q. & Basset, M. A. Training Feedforward Neural Networks Using Symbiotic. Hindawi Publishing Corporation Computational Intelligence and Neuroscience., pp. 1–14, 2016.

Singh, D. Y. & Chauhan, A. S. Neural networks in data mining. Journal of Theoretical and Applied Information Technology (JATIT)., pp. 37–42, 2009.

Guldal, V. & Tongal, H. Comparison of recurrent neural network, adaptive neuro-fuzzy inference system and stochastic models in egirdir lake level forecasting. Water Resources Management., vol. 24, no. 1, pp. 105–128, 2010.

Sibanda, W. & Pretorius, P., “Novel Application of Multi-Layer Perceptrons ( MLP ) Neural Networks to Model HIV in South Africa using Seroprevalence Data from Antenatal Clinics,†International Journal of Computer Applications., vol. 35, no. 5, pp. 26–31, 2011.

Mishra, S., Yadav, R. N. & Singh, R. P. A Survey on Applications of Multi-Layer Perceptron Neural Networks in DOA Estimation for Smart Antennas. International Journal of Computer Application., vol. 83, no. 17, pp. 22–28, 2013.

Gales, M. Multi-Layer Perceptrons. University of Cambridge Engineering Part IIB, pp. 1–39, 2015.

Martens, J., “Learning Recurrent Neural Networks with Hessian-Free Optimization,†in International Conference on Machine Learning (ICML), 2011, pp. 1033–1040.

Huang, B. Q., Rashid, T. & Kechadi, M. Multi-Context Recurrent Neural Network for Time Series Applications. International Journal of Computer Intelligence., vol. 1, no. 10, pp. 45–54, 2007.

Qin, Y., Song, D., Chen, H., Cheng, W., Jiang, G. & Cottrell, G. W. A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction. Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, pp. 2627-2633, 2017.

Ahangar, R. G., Yahyazadehfar, M. & Pournaghshband, H. The Comparison of Methods Artificial Neural Network with Linear Regression Using Specific Variables for Prediction Stock Price in Tehran Stock Exchange. International Journal of Computer Science and Information Security (IJCSIS)., vol. 7, no. 2, pp. 38–46, 2010.

Yu, X., Tang, L., Chen, Q. & Xu, C. Monotonicity and convergence of asynchronous update gradient method for ridge polynomial neural network. Neurocomputing, vol. 129, pp. 437–444, 2014.

Ghazali, R., Hussain, A. J. Liatsis, P. & Tawfik, H. The application of ridge polynomial neural network to multi-step ahead financial time series prediction. Neural Computing and Applications., vol. 17, no. 3, pp. 311–323, 2008.

Giles, C. L., Griffin, R. D. & Maxwell, T. Encoding Geometric Invariance in Higher-order Neural Networks. American Institute of Physics., pp. 301–309, 1988.

Yadav, R. N., Kalra, P. K. & John, J. Time series prediction with single multiplicative neuron model. Applied Soft Computing., vol. 7, no. 4, pp. 1157–1163, 2007.

Misra, B. B. & Dehuri, S. Functional Link Artificial Neural Network for Classification Task in Data Mining. Journal of Computer Science., vol. 3, no. 12, pp. 948–955, 2007.

Hassim, Y. M. M. & Ghazali, R. Functional Link Neural Network – Artificial Bee Colony Functional Link Neural Network-Artificial Bee Colony. In International Conference on Computational Science and Its Applications (ICCSA), 2013, pp. 24–27.

Akram, U., Ghazali, R. & Mushtaq, M. F. A Comprehensive Survey on Pi-Sigma Neural Network for Time Series Prediction. Journal of Telecommunication, Electronic and Computer Engineering., vol. 9, no. 3, pp. 57–62, 2017.

Shin, Y. & Ghosh, J. The pi-sigma network: an efficient higher-order neural network for pattern classification and function approximation. Proc. of the Seattle International Joint Conference on Neural Networks, IJCNN-91, pp. 1–18, 1991.

Yong, N. & Wei, D. A hybrid genetic learning algorithm for Pi-sigma neural network and the analysis of its convergence. Proc. of the 4th International Conference on Natural Computation, 2008, vol. 3, pp. 19–23.

Husaini, N. A., Ghazali, R., Nawi, N. M. & Ismail, L. H. The Effect of Network Parameters on Pi-Sigma Neural Network for Temperature Forecasting. International Journal of Modern Physics: Conference Series., vol. 9, pp. 440–447, 2012.

Carcano, E. C., Bartolini, P., Muselli, M., Piroddi, L., Montallegro, V. & Nazionale, C. Jordan recurrent neural network versus IHACRES in modelling daily streamflows. Journal of Hydrology., vol. 362, no. 3–4, pp. 291–307, 2008.

Husaini, N. A., Ghazali, R., Nawi, N. M. & Ismail, L. H. The Jordan Pi-Sigma Neural Network for Temperature Prediction. Ubiquitous Computing and Multimedia Applications., pp. 547–558, 2011.

Ghazali, R., Husaini, N. A., Ismail, L. H. & Samsuddin, N. A. An Application of Jordan Pi-Sigma Neural Network for the Prediction of Temperature Time Series Signal. INTECH Open Access Publisher., pp. 275–290, 2012.

Wysocki, A. & Åawry, M. Jordan Neural Network for Modelling and Predictive Control of Dynamic Systems. IEE Conference, Methods and Models in Automation and Robotics (MMAR)., vol. 2, no. 1, pp. 145–150, 2015.

Shin, Y. & Ghosh, J. Ridge Polynomial Networks. IEEE Transactions on Neural Networks, vol. 6, no. 3, pp. 610–622, 1995.

Waheeb, W., Ghazali, R. & Herawan, T. Ridge Polynomial Neural Network with Error Feedback for Time Series Forecasting. PLOS ONE, vol. 458, pp. 1–34, 2016.

Akram, U., Ghazali, R., H. Ismail, Zulqarnain, M., Husaini, N. A., and Mushtaq, M. F., “An Improved Pi-Sigma Neural Network with Error Feedback for Physical Time Series Prediction,†International Journal of Engineering & Technology., vol. 5, 2018.

Karamouz, M. S. N. & Falahi, M. Hydrology and Hydroclimatology: Principles and Applications. CRC Press. pp. 1-740, 2012.

Howarth, L. M., Roberts, C. M., Hawkins, J. P., Steadman, D. J. & Stewart, B. D. B. (2015). Effects of ecosystem protection on scallop populations within a community led temperate marine reserve. Marine Biology, vol. 162, no. 4, pp. 823–840, 2015.

International Council for the Exploration of the Sea. (1902). Retrieved on March 13, 1902, from https://en.wikipedia.org/wiki/International_Council_for_the_ Exploration_of_the_Sea.

Barnet, T. P., Pierce, D. W., Hidalgo, H. G. Bonfils, C. & Santer, B. D. Human-Induced Changes in the Hydrology of the Western United States. Science, pp. 1080-1083, vol. 319, no. 5866, 2008.

Dingman, S. L. Physical Hydrology. 3rd Edition. Waveland Press, 2015.

Wang, J. & Wu, J. Occurrence and potential risks of harmful algal blooms in the East China Sea. Science of the Total Environment, Science of The Total Environment, vol. 407, no. 13, pp. 4012–4021, 2009.

Ta-Yin, H. & Ho, W.-M. Travel Time Prediction for Urban Networks: The Comparisons of Simulation-based and Time-Series Models, Proc. of the 17th ITS World Congress (1), pp. 1–11, 2010

Huang, D.-S. & Jo, Kang-Hyun, L. W. Intelligent Computing Methodologies. Proc. of the Springer 10th International Conference on Intelligent Computing (ICIC), 8589, 2014.

Hussain, A. J. & Liatsis, P. Recurrent pi-sigma networks for DPCM image coding. Neurocomputing, vol. 55, no. 1–2, pp. 363–382, 2002.

Hussain, A. J., Liatsis, P., Tawfik, H., Nagar, A. K. & Al-Jumeily, D. Physical time series prediction using Recurrent Pi-Sigma Neural Networks. International Journal Artificial Intelligence and Soft Computing, vol. 1, no. 1, pp. 130–145, 2008.

Alarifi, A. S. N., Alarifi, N. S. N. & Al-Humidan, S. Earthquakes magnitude prediction using artificial neural network in northern Red Sea area. Journal of King Saud University - Science, vol. 24, no. 4, pp. 301–313, 2012.

Ramana, R. V., Krishna, B. & Kumar, S. R. (2013). Monthly Rainfall Prediction Using Wavelet Neural Network Analysis. Water Resource Manage, pp. 3697–3711, 2013.

Wang, J., Zhang, W., Li, Y., Wang, J. & Dang, Z. Forecasting wind speed using empirical mode decomposition and Elman neural network. Applied Soft Computing, 23, vol. 23, pp. 452–459, 2014.

Doucoure, B., Agbossou, K. & Cardenas, A. Time series prediction using artificial wavelet neural network and multi-resolution analysis: Application to wind speed data. Renewable Energy, vol. 92, pp. 202–211, 2016.

Radziukynas, V. & Klementavicius, A. Short-term wind speed forecasting with Markov-switching model. Proc. of the 55th International Scientific Conference on Power and Electrical Engineering of Riga Technical University (RTUCON) Short-term vol. 130, pp. 145–149, 2014.

Grigonytė, E. & Butkeviciute, E. Short-term wind speed forecasting using ARIMA model. Energetika, pp. 45–55, 2016.

Huang, Z. & Chalabi, Z. S. (1995). Use of time-series analysis to model and forecast wind speed. Journal of Wind Engineering and Industrial Aerodynamics, 56(2–3), vol. 56, no. 2–3, pp. 311–322, 1995.