Survey on Human Activity Recognition using Smartphone

The field of Human Activity Recognition (HAR) is an active research field in which methods are being developed to understand human behavior by interpreting features obtained from various sources, these activities can be recognized using interactive sensors that are affected by human movement. Sensor can embed elements within Smartphones or Personal Digital Assistants (PDAs). The great increase in smart phone users and the increase in the sensor ability of these smart phones, and users usually carry their smartphones with them. This fact makes HAR more important and accepted. In this survey, A number of previous studies were studied and analyzed, where we prepared a comparison of the research works conducted over the period 2010-2020 in human activity recognition using Smartphone sensors. Comparison charts highlight their most important aspects such as a type of sensor used, activities, sensor placement, HAR-system type (offline, online), computing device, classifier (type of algorithms) and system accuracy levels .


Introduction
Human Activity recognition has been an extremely practical research subject for the past two decades as a result of accelerometers and sensors, accessibility, reduced power consumption and low cost, artificial intelligence, machine learning, and the acceleration in the development of smart phone capabilities [1][2][3].
In the study of HAR, a number of human activities including running, sitting, walking, standing, sleeping, cooking, driving, showering, etc. are documented. Data is collected from wearable accelerometer or sensors, in addition to images or video frames. HAR is broadly used in areas such as crime control, medical diagnosis, controlling crime rates using monitoring, keeping track of old people, creating smarthome environment by daily activity recognition, recognizing driving activities leading to safe travel, recognizing Military actions … etc. [4][5][6]. In this paper, we present a proposal to design a system recognizing human activities using sensors founded in smart phones, focus on online recognition of activities and evaluate the performance of recognition of human activities by hybridization between one of the algorithms of the swarm intelligent with a deep learning algorithm (Convolution Neural Network -CNN) to achieve a computer system that identifies human activities efficiently and quickly.
Recognition of human activity using Smartphone sensors is the first topic of the next section of this study. A discussion of previous related works done by other researchers involving classifier algorithms for activity recognition will be tackled in section 3 followed by a conclusion.

Methodology
This comparison analysis is our starting point for finding a suitable and accurate algorithm based on cell phone accelerometer data for real-time human activity recognition. We use a smartphone for collecting accelerometer data, the raw data collected is then pre-processed. Hybridization between one of the algorithms of the swarm intelligent with a deep learning algorithm (Convolution Neural Network -CNN), for the classification and testing purpose. The results on the Table (2) summarize the algorithms that we have tested in this comparison study indicate that the best evaluation rate in our survey was 99.4%.

Recognition of Human Activity Using Smartphone Sensors
Human Activity Recognition (HAR) is a significant aspect of research in machine learning which represents the task of measuring a person's activity via the use of objective technology. The HAR research can be done with the help of Smartphone, images or sensors. Today smartphone can be used as a full HAR system the need to add extra mechanisms [7]. At the same time as smartphones are becoming an incorporated part of the human's daily life for its ability to do intricate computation, internet connection and include vast quantities of hardware sensors. Therefore, they can provide new opportunities in the HAR research [8][9][10]. Putting in plain terms, it includes the use of diverse sensing technologies to collect and categorize user activities in different domains, starting from medical applications, home monitoring & assisted living, to sports and leisure applications [11][12][13][14][15]. The human activity to be studied can be identified by using various sensors placed on the individual's body. The sensor(s) plays an important role in the detection of human behavior in a traditional HAR, The sensor(s) collects the data obtained from the gesture of the human body and the recognition engine analyzes the data and decides the type of action performed, In order to extract features, the raw data collected is then pre-processed, and the data is categorized to accurately identify certain activities. A wearable technology, this involves all body-worn devices that capture and process user information and their contact with the environment. A Smartphone as a wearable device, numerous internal sensors are now provided, some of which can be used for motion sensing and are therefore ideal for the detection of human activities. A list of these sensors like Accelerometer, Gyroscope, Camera, GPS, Barometer). They provide information about the user's linear acceleration and angular velocity respectively when used as a wearable sensor, and are not highly affected by external factors such as bad indoor signal reception in GPSs or electromagnetic noise in compasses [2][16] [17].

Human Activity Recognition
The Human Activity Recognition procedure consists of four core phases as shown in figure1.These phases are: data collection, feature extraction, classification and recognition activities [18]. HAR models take the reading of the basic sensor data as an input and their output are the predictions of the user's movement activities [19]. activities in order to collect data. Important sensors used to recognize human activity are: accelerometers which are used to measure acceleration that represents the rate of change in the velocity of an object; gyroscopes which are used to measure orientation and angular velocity; Camera captures Images and video ; Compass Measures orientation respect to the magnetic north; GPS Tracks global 3D positioning directly; Barometer can provide altitude coordinates and helps to obtain GPS position quickly ;motion sensors (or motion detector) use one or more techniques to detect motion in an environment; biosensors is an analytical instrument for decomposing matter detection.; pressure sensor is a device for measuring the pressure of gases or liquids; proximity sensor, a sensor without any physical contact, can detect the presence of objects near it [20][21][22][23][24].

Extracting Features:
One of the important steps in the identification of human activity is the process of extracting features, since this stage provides important information that helps in the process of distinguishing between different activities. The features in both Time and Frequency domains are extracted [25]. Simple timedomain features like autocorrelation, variance, zero-crossing rate, mean, etc. are used for activity recognition with the help of Smartphone sensors [26].

Classification process:
In the stage of classification, machine learning tools are used for the classification of collected data, Machine learning tools are artificial intelligence algorithmic applications that offer systems the ability to learn and evolve without adequate human feedback. There are four types of machine learning algorithms:  Supervised: learning builds predictive models using classification and regression techniques. Support vector machine (SVM), k-nearest neighbor, and Naïve Bayes, are popular algorithms for performing classification. Linear model, nonlinear model, neural networks, and adaptive neuro-fuzzy learning include common regression algorithms.  Unsupervised: The most prevalent unsupervised learning strategy is clustering.
Common algorithms for performing clustering include k-means clustering , hierarchical clustering, PCA, and Gaussian mixture models.  Semi-Supervised: In order to classify data assets, semi-supervised learning applies the classification process and the clustering process to group them into different sections. Semi-Supervised Learning Algorithms is Graph-Based Algorithms and Multi-view Algorithms.  Reinforced: The most prevalent Reinforced learning technique is classification.
Popular algorithms for performing classification are Q-learning, R-learning and TD-learning. Figure (2) illustrate types of machine learning algorithms. To classify data, we can apply a single classifier or a combination of classifiers on dataset both in offline and online modes. Classification comprises training and testing where training is a preparation step which aims to obtain the model parameters. After training the classifiers, testing is done to ensure whether the activities are correctly identified or not [27][28][29]. 4. Recognition Activities: In this step, classification-trained models are used to classify data collected from the first one to classify human activities [30][31].

Human Activities
The HAR system depends on the set of activities to be recognized as they can directly affect the way systems are designed and implemented [32] [33]. The activity itself is a sequence of actions while recognition is the ability to determine something from prior knowledge, sensing or achievement [34].

Human Activity Recognition System Type:
HAR systems can be classified into two main groups with respect to the time response systems take to perform the classification [35] [36].
 Online methods mean that the classifiers are trained on the hosting device such as a real-time mobile cloud. The raw data are not stored for later use but, instead, they are immediately processed for training in order to save time.  Offline methods mean using a desktop machine for training the classifiers beforehand. They simply do not require real-time operation. The raw data is stored and in later time these data are used for training the classification model [37][38].

Related literature in the HAR Systems
A lot of researches have proposed HAR for the recognition of human activities. They include a number of application areas such as smart homes, healthcare, security, surveillance etc.
HAR systems are capable of sensing, monitoring, and learning from human actions as they offer useful information which can improve decisions about human future needs or behavior. Works that will be discussed and studied in this survey are as follows: Davide Anguita et al. [39] offered a special system on human physical activity using smart phone sensors. Experiments' population comprises a group of 30 volunteers between 19 to 48 years. Each person performed six activities: standing, lying down, walking, walking upstairs and walking downstairs, while wearing a Smartphone on the waist and employing SVM as classifier. Samsung Galaxy S2 Smartphone has been used in the testing.
[32a] suggested an HAR system that merged the internal accelerometer of the smartphone with an external one for the classification of 10 activities on the user's wrist.
kwapisz et al. [40] developed a new system that uses smartphones to perform activity recognition. To apply their system, they collected data from twenty-nine users doing daily activities by putting cell phones in their pockets. They used Decision Tree (DT) and K-Nearest Neighbor (KNN) to categorize 6 activities namely sitting, and standing, walking, jogging, climbing stairs.
Mustafa Kose et al. [16] projected an activity recognition system working on Android platforms that supports online training and classification while using only the accelerometer data for classification. They evaluated the performance of activity recognition using the Naïve Bayes classifier and K-Nearest Neighbor (KNN) classification algorithms, called Clustered KNN.
Wu et al.
[32b] A combination accelerometer and gyroscope system was used for the classification of 9 activities using the iPhone 4. They showed insight into the benefits of integrating gyroscope signals into the recognition system, achieving classification accuracy increases of between 3.1 percent and 13.4 percent.
Rodriguez-Martin et al.
[32c] they introduced a way for patients to differentiate between stand-to-sit and sit-to-stand. Via measurements from a waist-based triaxial accelerometer. For classification they used Threshold-based mechanism and Support Vector Machine (SVM) .
Bruno et al .
[32d] to identify 8 activities used Gaussian Mixture Model (GMM) and Gaussian Mixture Regression (GMR) as classifier. Davide Anguita et al. [41] Introduced a new publicly accessible dataset for HAR using Smartphone and recognized some results using multi class Support Vector Machine approach.
Amin Rasekh et al. [42] suggested a powerful activity recognition schemes that depend on a Smartphone. The scheme uses a 3-dimentional smartphone accelerometer as the only sensor. The authors used a smartphone placed in the region of waists such as jacket pocket or pants pocket. To classify 5 activities Quadratic Classifier, K-Nearest Neighbor Algorithm, Support Vector Machine (SVM), and Artificial Neural Networks are used. The best categorization rate in their research was achieved by SVM.
Girija Chetty et al. [43] presented a modern method for intelligent recognition of human activity (AR). Using Naïve Bayes (NB), Decision Tress (DT), Random forests (RF), Random committee (RC), and lazy learning (IBk) as classifiers to recognize six common activities: walking, walking upstairs, walking downstairs, sitting, standing and laying in real time.
Ali M. Muslim et al. [44] presented a new approach that incorporated a smart watch, fastened to the human's ankle, and another smartphone freely carried by the user. They used threshold-based mechanism to classify 20 different human activities. This process serves the job of distant real-time inspection of the user's human activities. Shugang Zhang et al. [45] using a smart watch, designed real-time activity recognition and counting procedure. The Support Vector Machine classifiers (SVM) were trained to identify nine activities in real-time.
M.Son Dao et al. [38] a new approach to understand daily human activities using Symbolic Aggregate Approximation SAX-based features and adaptive learning was presented.
Chun-Ting Chen et al. [30] developed a smartphone model system. They accomplished a sequence of researches to estimate the performance of four popular classification methods on six activities, including standing, sitting, walking, jogging, biking, and climbing stairs. The system used Multi-layer Perception (MLP), Decision Tree (J48), Random Forest (RF) and Instance-based K-Nearest Neighbor (IBK) as classifiers.
Erhan et al. [46] offered a model a smartphone carried by disparate man and women who perform six activities walking, climbing up the stairs, climbing down the stairs, sitting, standing and laying. They classify the activities by using different machinelearning approaches such as Decision Trees, Support Vector Machines and K-Nearest Neighbors (KNN).
Ahmed Younes Shdefat et al. [47] suggested a method for observing twelve human activities (ADL) such as sitting, laying, standing, attaching to table, walking, jogging, running, jumping, pushups, stairs down, going up stairs, and cycling. The twelve activities were divided into two groups: the first group comprises six of them with Support Vector Machine (SVM); and other six with K-Nearest Neighbor (k-NN) .
Roobini et al. [48] designed a HAR system of two forms: the first form used CNN with LSTM; the second form is organized as RNN with LSTM. The dataset is preprocessed using noise filters and loaded by using three main types of sensors namely total acceleration, body acceleration and gyroscope. Then, Convolutional Neural Network with Long-Short Term Memory (ConvLSTM) and Recurrent Neural Network with Long-Short Term Memory (RNNLSTM) is used as classifiers. The performance of the model is validated on the basis of Accuracy.
Robert et al. [49] present a human physical activity recognition system depending on data obtained from smartphone sensors. The procedure entails developing a classifier using three sensors accessible on a smartphone: accelerometer, gyroscope, and gravity sensor. For their proposition, they target walking, running, sitting, standing, ascending, and descending stairs. Results show good accuracy for recognizing all six activities. The system is fully implemented on a mobile device as an Android application.
Mustafa Badshah [50] proposed the general architecture which is utilized to build human activity recognition systems using Decision Tree, Random Forrest, Artificial Neural Network, and Recurrent Neural Network to classify 6 activities namely walking, walking upstairs, walking downstairs, sitting, standing and laying. The present study attempts to show a case of reduction in computational cost and significant achievement in accuracy using methods of feature selection. Nadeem Ahmed et al. [20] presented a hybrid feature-choice technique, which includes a filter and a covering method. The procedure uses a Sequential Floating Forward Search (SFFS) to gain preferred features for improved activity recognition. After that they input the features into a multiclass Support-Vector-Machine (SVM) to generate nonlinear classifiers by implementing the kernel trap for training and testing function.

Results
A survey has been done on twenty papers published between 2010-2020 for the recognition of human activity using smartphone sensors. The criteria used to categorize these procedures are: sensor-type which relies on the class of signals measured to extract activity information; sensor placement in the body of the user; type of human activity by which systems are grouped with respect to the activities they are able to identify (e.g. locomotion and ADLs); the learning procedure which is related to the type of algorithms used for learning methods such as supervised, semi-supervised or unsupervised methods. Table (2) provides a comparison of the systems discussed. The comparison shows which criteria were used to categorize these procedures.

Discussion
Human Activity Recognition (HAR) is a significant aspect of the research in machine learning which represents the task of measuring person's activity via the use of objective technology. The state-in-the-art of human activity identification is discussed in this paper. The general structure of the online and offline activity recognition system, traditional and deep learning machine learning algorithms, was mentioned. In addition, those studies concentrate on understanding the amount of activities used for the recognition process and various classification methods. In terms of the operations, devices used, learning models, dataset, and accuracy of recognition, twenty studies are qualitatively compared. Lastly, we discuss the numerous difficulties and problems of these studies.

Conclusion
This survey has shown that deep learning has been used more recently than conventional machine learning and in proposed work to recognized human activities we used machine learning. The results on the Table (2) summarize the algorithms that we have tested in this comparison study indicate that the best evaluation rate in our survey was 99.4%. For future work, we want to improve our role of activity recognition. First We want to recognize additional activities. Second we want to collect data from more humans of different ages, third we want to obtain more functionality that could differentiate between different activities better by hybridization between one of the algorithms of the swarm intelligent with a deep learning algorithm (Convolution Neural Network -CNN).