Edge Impulse Potential to Enhance Object Recognition Through Machine Learning

Pinaso, G. R.; Figueiredo, L. M.; Rosa Júnior, O.; Richetto, M. R. S.

DOI 10.5433/1679-0375.2024.v45.49197

Citation Semin., Ciênc. Exatas Tecnol. 2024, v. 45: e49197

Received: 24 October 2023; Received in revised for: 9 May 2024; Accepted: 28 May 2024; Available online: 20 June 2024.

Abstract:

Machine Learning (ML) is a powerful artificial intelligence branch that can help businesses, whether small or large, in a variety of industries. It is an option for replacing resources with high operating costs. The aim of this study was to use the Edge Impulse platform as an ML tool option. The system applies low-code frameworks which abstracts a series of complex techniques applied in ML, such as data processing and AI components structure. It implies a time reduction during the development period. Using Edge Impulse allows a more user-friendly interface alternative with an easy-to-interpret logic flow. The study focused an application to do an object recognition, aiming the system capacity limit. The autogenerated accuracy value, pointed by the system indicated 97.9 % after the training step and 89 % after retesting the first 20 mices, photographed in different image angles, indicating a possible model overfitting. Even though, the system showed promise in terms of classifying objects. Some adjustments in the image dataset can improve the model capacity of recognition, as the amount of images showed insufficient at the survey’s conclusion.

Keywords: artificial intelligence, machine learning, object recognition, low-code applications

Introduction

In the midst of the 21\(^{st}\) century, it has become crucial for big or small companies in any industry to implement innovative resources that have a positive effect on product quality, responsiveness, and customer service in order to remain competitive. Market segments are drawn to the technological race because it enables them to increase their production efficiency.

Supported by the implementation and growth of Lean Six Sigma and Lean Manufacturing thinking, intelligence technologies provide professionals with powerful digital tools. This possibility allows examining and evaluating operations without relying on inefficient and laborious practices in sectors such as quality and processes, as Tarantino, 2022 argues. This line of thought leads to the development and utilization of Artificial Intelligence (AI), which, according to Taulli, 2020, represents a significant turning point in history. Unlike the innovative technologies of the 19th and 20th centuries, AI stands out because it does not require a high-cost physical infrastructure to be adopted. Its benefits can be accessed through existing hardware that can be carried in a pocket, like smartphones.

Gaps and challenges demonstrate how much AI and robotics systems need to evolve in ethical and governance terms (Winfield & Jirotka, 2018). The authors treated as essential building trust over Autonomous intelligent Systems (AIS). An argument for this is the fact that ML is already being used to improve customer experiences, boost innovation, and efficiently reduce costs. As this, the authors proposed five pillars for IAS ethical aspects.

ML is now pervasive in nearly all human endeavors for innovation, regardless of the industry they are tied to (Diego Carou et al., 2022). Its influence can be observed in commonplace applications that are integrated into daily life, including streaming services which provide recommendations based on a user’s viewing history as well as mobile phone photograph software that sorts albums utilizing facial recognition technology or behavioral habits of customers on e-commerce (Jain et al., 2023). Finally, machines using AI technologies in more complex company applications can prevent defective parts and products (Cioffi et al., 2020) from being shipped (Pournader et al., 2021), reduce accidents (Xuecai et al., 2019), and provide statistical forecasts for the production process, among other benefits.

Preliminaries and problem statement

Due to an increasing demand for innovative solutions and the utilization of advanced technologies, the popularity of ML has grown, and its application has expanded. The primary objective is to enhance process control and efficiency while pursuing excellence and improving responsiveness.

Regarding quality control in manufacturing areas, various points in processes or products may necessitate inspections. The proficiency and experience levels of operators are crucial in this domain. According to Calixto et al., 2022 findings, the logistics industry is characterized by intricate requirements due to the significant volume of people and services involved. The author utilizes artificial intelligence for cargo management, utilizing the OpenCV library and Python programming language.

Moreover, Sundaram & Zeid, 2023 and [Mer - not found] establish the possibility of incorporating machine learning techniques in the visual examination of castings. In this instance, the researchers utilized the YOLO framework, which is a modern and efficient architecture for detecting objects. Machine learning proved to be a feasible means of examining and categorizing peaks in stamped components, as confirmed by Block et al., 2021. The system implemented both the VGG16 and Resnet convolutional neural network models.

Bilik & Horak, 2022 analyzed image characteristics using Scale Invariant Feature Transform (SIFT) and Speed-Up Robust Feature (SURF) descriptors, and employed automatic decision techniques using Support Vector Domain Description (SVDD) and Support Vector Machine (SVM) classifiers. In a similar vein, Mezavila et al., 2021 utilized machine learning (ML) to analyze heartbeats in the health sector, implementing the scikit-learn, TensorFlow, and Keras libraries with the Python programming language.

Dalzochio et al., 2020 state that enumerating the various tools, applications, software, platforms, and methods developed for machine learning is a challenging task. Python programming language is widely accepted as the preferred language in Computer Science, Data Science and ML as highlighted by Raschka et al., 2020. One major challenge in developing ML solutions is the amount of data required for training. For some applications, like deep Learning, a huge dataset can generate better results (Klein & Bergmann, 2018).

Aidora Abdullah & Tan Yeong Yih, 2014 approached the field of computer science education as a significant challenge, emphasizing the multitude of extrinsic and intrinsic factors that influence both learners and educators. Disciplines such as Computer Programming, Image Processing and Applications development inherently present complexities to certain students, using an approach that balances challenge and accessibility. There must be a collaborative relationship between educators and learners to effectively navigate these complexities, creating environments that stimulate curiosity, foster critical thinking, and offer appropriate support to enhance engagement and facilitate mastery. The same way, for those who are unfamiliar with AI and its requirements, it’s complex to apply ML. There are obstacles to overcome as they work on developing ML applications. An individual’s ability to learn and adapt to the mentioned technologies is essential for their completion.

Thus, this study aims to explore the following research question: To what extent does the Edge Impulse platform facilitate the development of AI applications? An investigation into its efficacy can be achieved by creating a test application using this low-code platform.

The primary objective of this research endeavor was to design and implement an Edge Impulse AI Application utilizing a predictive model. The development process was undertaken by a team of novice programmers, guided by the stipulation that the application be constructed with a low-code paradigm, minimizing or eliminating the necessity for direct interaction with traditional programming languages. Subsequently, the study undertook an evaluation to assess the predictive efficacy of the model generated within the Edge Impulse platform.

To achieve the overall objective, specific goals were established:

• Creating a system that employs ML for object detection.

• Training the ML tool.

• Analyzing the obtained results.

The Edge Impulse platform, founded in 2019 by Zack Shelby and Jan Jongboom, has been lauded for its innovation and development of edge AI tools. The platform has received recognition as the best innovation of 2021, the best edge AI development tool of 2022, and as one of the top four edge computing vendors (Mihigo et al., 2022). The mission of the platform, according to its founders, is to offer developers the means to create intelligent devices that will have a positive and impactful effect on society and industry through the use of machine learning.

Edge Impulse is a cloud-based platform for ML operations that enables the development of embedded and edge systems. These systems can be employed across a diverse range of hardware (Hymel et al., 2023).

Edge devices are a diverse range of electronic devices, including smartphones, tablets, laptops, and personal computers (Schulz et al., 2023). Edge Impulse aims to simplify computer science operations and increase the accessibility of machine learning to the general public by allowing easy model optimization and deployment on any edge device. This simplifies the process of creating, deploying, and scaling machine learning applications, making it faster and more efficient. The development of smarter devices by developers will potentially enhance various sectors (Cınar et al., 2020).

ML has three primary types of learning, as described by [Lop - missing information]: supervised, unsupervised, and reinforcement learning. It should be noted that this study only focuses on supervised learning, which requires directing the algorithm’s learning using labeled data and examples. The application must undergo supervised training by a human, who defines correct and incorrect responses. This enables the algorithm to learn the properties that distinguish each label (Sarker, 2021). After training, the application is tested in unsupervised scenarios to assess accuracy, or the hit rate, when no supervision is present.

Numerous supervised learning algorithms have been proposed in the literature. Mahesh, 2020 have classified most of them as either classification or regression. Classification’s purpose is to classify inputs, while regression’s purpose is to predict an output. Each algorithm is useful in distinct scenarios. For instance, classification can distinguish whether an image is of a dog or not. The second methodology is capable of predicting the temperature in a city utilizing the collected variables.

During the research, was observed that other authors had focus into the applications programming language, libraries, tools and frameworks to implement the ML algorithms. This research contributed to the academics differently.

The main focus was to use an application codeless to implement the ML functions, with any interaction with the system’s language. The effectiveness of the application was the main target, to know if some AI tools can be used by non-IT professionals.

Materials and methods

The study aims to evaluate the effectiveness of utilizing Edge Platform , even among users lacking knowledge of programming languages.

To confirm this research hypothesis, we undertook the subsequent steps to develop the application:

• Defining the data type to be captured.

• Formation of a data set, consisting of both training and test data, involved uploading images collected by a smartphone.

• Validation of the results.

• Pre-processing.

• Data interpretation with the application.

The research used one of the Edge Platform resources: object recognition. The image dataset was used with different kinds of electronic devices, such as computer mouses and smartphones. The study adopted a qualitative approach to measure the object recognition capabilities of Edge Impulse and to develop applications using the platform. This is an office tool that comes in multiple models for everyday use. The objective was to validate the device’s efficiency and consistency compared to the industry benchmark.

In order to do not exceed the free trial limits provided by Edge Impulse platform, avoid a long data analysis waiting time, it, we utilized 100 images of the same mouse, each taken from various positions, angles, and distances. To assist Edge Impulse in interpreting the database and generating the model, we standardized photograph sizes and positioned them in portrait mode. Of the complete set of photos, 80 were used for training and 20 for testing, aligning with the platform’s recommended proportion. To test the platform wizards, the study didn’t adopt any kind of cross-validation or hold out technique.

Within the Edge Impulse environment, we extracted the data features and applied a learning block for classification of new data. To facilitate this process, we labeled the mouse-relevant parts of the images and defined the scale, as well as the number of cycles per photo for training and the learning level. To assess the object detection capabilities of objects dataset and the identification of false positives, another experiment was conducted. Another sample of 12 types of mouses, a remote control, a calculator, glasses case, notebook power supply and smartphone cover were photographed for use in a new dataset. The similarity of the devices was a challenge that the research was trying to test during the experiment. The results of image recognition were also compared. Image classification was achieved by uploading photos to the platform and classifying them in real-time. An edge device was positioned for local processing and equipped with a camera to scan objects.

Results

Using the initial dataset, consisted of pictures of the same mouse as stated earlier, we followed the logical structure of the platform after establishing the classification model.

Subsequently, one stage of the platform provided us with a graphical analysis. Utilizing the image features, the system could differentiate the varying photo angles and positions. Figure 1 presented the graph of similar photo angles in two groups. However, all images were classified as mouse by the system. Nonetheless, Edge platform made automatically the dispersion graph, where two groups of different mouse angles were shown.

[fig1]

After training, the platform presents to the user, an automatic accuracy calculation generated according to data analysis done. In this case, system achieved an accuracy rate of 97.9% and it was automatically calculated by Edge Impulse. It was a value showed to users at application’s main screen. After this result, the application was tested manually by the developers. Against the initial 20 photos used in training, the model correctly classified all 20 new images of the same objects, in different image angles with a certainty level exceeding 89%, indicating a high degree of accuracy in identifying mouses in all photos.

An additional test was conducted using 12 images of mouses of various brands, models, and colors, as illustrated in Figure 13. The images were presented to the system to verify the model’s effectiveness and evaluate its accuracy.

(a)	(b)		(g)	(h)
(c)	(d)		(i)	(j)
(e)	(f)		(l)	(m)

[fig2]

Figure 13 shows the results of this second test, in which only one image from the second dataset was incorrectly classified as a mouse by the tool, as depicted in Figure 13(a). The remaining images were classified with high accuracy, achieving a rate of 91.67 % during the test, as shown in Figures 13(b)-(m). It’s relevant to remember that one of this research’s objectives was to test the Edge Impulse codeless infrastructure and resources. Because of this, the authors developed the additional test.

In order to identify false positives, or objects that do not follow the pattern, an additional experiment was conducted involving 8 objects. The purpose was to compare the analysis procedure of uploading photos one at a time versus real-time detection using an edge device. Among the images, only 2 mouses were utilized, along with 6 other objects that had colors and shapes compatible with the mouse, which were used to train the ML. A classification test was then performed using four different photos of each object. The objects were positioned at various angles to analyze the model’s potential incorrect interpretation.

Results were organized in Table 1. The application was expected to produce a specific outcome for each object. For the mouses, positive recognition was anticipated in both sets of 8 photos. For the remaining objects, recognition was not expected to occur. The test results are displayed in the classification column of Table 1, with the following column indicating the model’s level of certainty that the tested object was a mouse.

0.165cm

Based on the latest test, the results of readings taken via photo upload and scanning did not differ significantly. The training applied to both scenarios without any notable variation.

To conclude the tests, an additional experiment was conducted to re-examine false positives in the classifications. Various types of objects were submitted to the machine learning system in this experiment, including Knup and Multilaser mouses, as well as a smartphone case, as depicted in Figure 17. These objects were grouped together to evaluate the system’s response when subjected to simultaneous analysis.

(a)		(b)
(c)		(d)

[fig3]

As shown in Figure 17(a)-(d), the level of certainty remained unstable. This result was measured by the application inserting a value between 0 and 1 near the detected object. By this, the smartphone cover was either not recognized or validated as a false positive. On the other hand, the mouses were accurately classified at all times and with a high level of certainty.

Conclusions

The Edge Impulse platform showed promise in terms of classifying the tested objects. Figure 13 indicates that only one image, with a different mouse angle image, was misclassified, prompting consideration of augmenting the data set with images of other mouse models. This approach would help capture similar characteristics across different models of equipment, ultimately improving the system’s accuracy.

The tool seems feasible for categorizing objects and can be relevant in multiple domains. In addition, after examining false positives, it is advisable to evaluate the exposure duration of objects to the camera to decrease false positives. However, for intricate tasks, further exploration is necessary. In addition, after examining false positives, it is advisable to evaluate the exposure duration of objects to the camera to decrease false positives.

The demonstration of Edge Impulse as a viable option for individuals lacking proficiency in ML structures, programming languages, and related resources has surpassed expectations. Accurate application implementation was feasible even without extensive knowledge of aforementioned areas.

For optimal performance within Edge Impulse, future studies should consider implementing established frameworks or algorithms for building ML systems.

Author contributions

G.R. Pinaso and L.M. Figueiredo participated in conceptualization, data curation, formal analysis, funding acquisition, investigation, methodology, projects managements, resources, visualization, writing – original draft. O. Rosa Júnior and M.R.S. Richetto participated in supervision, validation, visualization, writing – revision and editing.

Conflicts of interest

The authors certify that there is not a commercial or associative interest that represents conflict of interest in relation to the manuscript.

References

Aidora Abdullah & Tan Yeong Yih (2014). Implementing Learning Contracts in a Computer Science Course as a Tool to Develop and Sustain Student Motivation to Learn. 256-265. https://doi.org/10.1016/j.sbspro.2014.01.1422

Bilik, S. & Horak, K. (2022). SIFT and SURF based feature extraction for the anomaly detection. ArXiv, 2, 1-7. https://doi.org/10.48550/arXiv.2203.13068

Block, Sylvio Biasuz, da Silva, Ricardo Dutra, Dorini, Leyza Baldo & Minetto, Rodrigo (2021). Inspection of Imprint Defects in Stamped Metal Surfaces Using Deep Learning and Tracking. 68(5), 4498-4507. https://doi.org/10.1109/TIE.2020.2984453

Calixto, J. M. T., Corrêa, M. S. & De Oliveira, M. A. (2022). A empregabilidade da inteligência artificial na automação do setor logístico para controle de carga. 1-30.

Diego Carou, Antonio Sartal & J. Paulo Davim (2022). Machine Learning and Artificial Intelligence with Industrial Applications: From Big Data to Small Data. Springer. https://doi.org/10.1007/978-3-030-91006-8

Cınar, Z. M., Nuhu, A. A., Zeeshan, Q., Korhan, O., Asmael, M. & Safaei, B. (2020). Machine learning in predictive maintenance towards sustainable smart manufacturing in industry 4.0. 12(19), 1-42. https://doi.org/10.3390/su12198211

Cioffi, R., Travaglioni, M., Piscitelli, G., Petrillo, A. & De Felice, F. (2020). Artificial intelligence and machine learning applications in smart production: Progress, trends, and directions.. 12(2), 1-26. https://doi.org/10.3390/su12020492

Dalzochio, J., Kunst, R., Pignaton, E., Binotto, A., Sanyal, S., Favilla, J. & Barbosa, J. (2020). Machine learning and reasoning for predictive maintenance in industry 4.0: Current status and challenges. 1-15. https://doi.org/10.1016/j.compind.2020.103298

Hymel, S., Banbury, C., Situnayake, D., Elium, A., Ward, C., Kelcey, M., Baaijens, M., Majchrzycki, M., Plunkett, J., Tischler, D., Grande, A., Moreau, L., Maslov, D., Beavis, A., Jongboom, J. & Reddi, V. J. (2023). Edge impulse: An mlops platform for tiny machine learning. ArXiv, 3, 1-15. https://doi.org/10.48550/arXiv.2212.03332

Klein, Peter & Bergmann, Ralph (2018). Data generation with a physical model to support machine learning research for predictive maintenance.

Jain, Varsha, Wadhwani, Kunal & Eastman, Jacqueline K. (2023). Artificial intelligence consumer behavior: A hybrid review and research agenda. 23(2), 676--697. https://doi.org/10.1002/cb.2233

LoPiano, Stefano Ethical principles in machine learning and artificial intelligence: Cases from the field and possible ways forward. 1--7. https://doi.org/10.1057/s41599-020-0501-9

Jain, V., Wadhwani, K. & Eastma, J. K. (2018). Data Generation with a Physical Model to Support Machine Learning Research for Predictive Maintenance. https://doi.org/10.1002/cb.2233

Klein, P. & Bergann, R. (2018). Artificial intelligence consumer behavior: A hybrid review and research agenda.

Lo Piano, S. (2020). Ethical principles in machine learning and artificial intelligence: cases from the field and possible ways forward. https://doi.org/10.1057/s41599-020-0501-9

Mahesh, B. (2020). Machine learning algorithms a review. 9(1), 381--386. https://doi.org/10.21275/ART20203995

Mery, D. (2020). Aluminum casting inspection using deep learning: A method based on convolutional neural networks. 1-12. https://doi.org/10.1007/s10921-020-0655-9

Mezavila, S. A., Dias, A. A. & Franco, M. E. (2021). Aprendizagem de máquina aplicada a análise de batimentos cardíacos. 8(1), 1-14. https://doi.org/10.18406/2359-1269v8n12021191

Mihigo, I. N., Zennaro, M., Uwitonze, A., Rwigema, J. & Rovai, M. (2022). On-device iot-based predictive maintenance analytics model: Comparing tinylstm and tinymodel from edge impulse. 22(14), 1-20.. https://doi.org/10.3390/s22145174

Pournader, M., Ghaderi, H., Hassanzadegan, A. & Fahimnia, B. (2021). Artificial intelligence applications in supply chain management. 1-16. https://doi.org/10.1016/j.ijpe.2021.108250

Raschka, S., Patterson, J. & Nolet, C. (2020). Machine learning in python: Main developments and technology trends in data science, machine learning, and artificial intelligence. 11(4), 1-44. https://doi.org/10.3390/info11040193

Sarker, I. H. (2021). Machine learning: Algorithms, real-world applications and research directions. 1-21. https://doi.org/10.1007/s42979-021-00592-x

Schulz, A., Stathatos, S., Shriver, C. & Moore, R. (2023). Utilizing online and open-source machine learning toolkits to leverage the future of sustainable engineering. 1-13. https://doi.org/10.48550/arXiv.2304.11175

Sundaram, S. & Zeid, A. (2023). Artificial intelligence based smart quality inspection for manufacturing. 14(3), 1-19. https://doi.org/10.3390/mi14030570

Tarantino, A. (2022). {\textnormal{Smart Manufacturing:. John Wiley \& Sons. https://doi.org/10.1002/9781119846642

Taulli, T. (2020). Introdução à Inteligência Artificial: uma Abordagem Não Técnica. Novatec Editora.

Xuecai, X., Gui, Fu., Yujingyang, X., Zigi, Z., Ping, C., Baojun, L. & Song, J. (2019). Risk prediction and factors risk analysis based on IFOA-GRNN and apriori algorithms: Application of artificial intelligence in accident prevention. 169-184. https://doi.org/10.1016/j.psep.2018.11.019

Winfield, A. F. T. & Jirotka, M. (2018). Ethical governance is essential to building trust in robotics and artificial intelligence systems. 376(2133), 1-13. https://doi.org/10.1098/rsta.2018.0085