Key software features

Ultron’s software-defined architecture enables the application of software logic through a centralized interface, allowing for greater flexibility in the development and deployment of autonomous infrastructure. The key features of the software include:

On-device data acquisition and control software

• User authentication to prevent unauthorized access

• Automatic registration and display of physically connected modules on the UI

• Allow real-time querying and status monitoring through visual representation of control blocks with feedback mechanism on the user interface

• Save the current configuration or load a previously saved configuration.

• Emulate analog and digital I/O and provide access to logic elements such as timers, counters, and control bits to meet your needs

• View deployed software configuration, communication settings, general information, and firmware versions of the device

• Modify parameter values in the parameter editor to control local or remote I/O

• View health metrics such as GPU, CPU performance, and average temperature of the compute unit

Flow-based visual programming tool

• A low-code integrated development environment for implementing logic on PLCs and automating processes

• A drag-and-drop interface for controlling inputs and outputs without having to know about underlying programming concepts

• Implement custom nodes to interact with Ultron’s analog and digital GPIO

• Wire automation flow easily using a wide range of communications protocols and function nodes

Gstreamer plugin and vision SDK

• Add and configure RTSP streams using the UI

• Integrate NVIDIA DeepStream and Gstreamer video analytics elements and control output signals based on inference result using customized SDKs

• Simplify the process of vision system integration by removing the need for specialized tools or programming languages to apply image processing logic

• Deploy and concurrently run multiple AI models in each module using the modular software design

Visualization and HMI

• Create easy to understand visualization and interactive elements that are mapped to actual values and controls

• Display real-time data and visualize inference results on a single centralized dashboard

• Allow operator to monitor and interact with process control systems to improve production efficiency

• Automate and display visual alarms on the HMI to ensure safety and security

System Architecture

Data collection and system supervision

Ultron is powered by NVIDIA Jetson Linux4Tegra (L4T), a Linux-based embedded operating system developed by NVIDIA for all Jetson platforms and supported by NVIDIA Jetpack SDK, a comprehensive set of libraries for acceleration of GPU computing, multimedia, graphics, and computer vision. Benefits of using Linux as a platform include background logging of critical operations and support for open communication standards such as SSH for secure communications.

A centralized supervisory software that monitors and controls the entire platform sits on top of the Jetson-based PLC hardware. The software processes collected data from inputs and sends commands to control output processes. Because the device's software is cloud-native, the containerized app can be easily managed and deployed at scale.

GPU-accelerated computing with Ultron

The collected data is useful for AI model training and for developing seamless streaming pipelines to extract meaningful real-time insights. The NVIDIA DeepStream SDK is a comprehensive analytics toolkit for AI-based multi-sensor processing. The toolkit is based on the GStreamer multimedia framework and includes a GPU-accelerated plug-in pipeline for building end-to-end AI-powered applications that analyze video and sensor data from connected inputs.. By leveraging Jetson’s high computing power and the unified SDK, , DeepStream and GStreamer elements can be easily integrated into the automation system and shorten the time required to build and deploy real-time Intelligent Video Analytics (IVA) applications. For example, engineers can build end-to-end DeepStream pipelines to quickly convert raw video input data into insightful annotated video output.

AI Inference at the edge

Aside from increased computation capability, real-time inference at the edge reduces latency significantly. This is vital when connectivity is unavailable, such as with remote devices, or when the latency to send data to and from a data center is too long. Edge AI minimizes data transfer between edge devices and their data centers. There is also more privacy and security by storing the data locally.

Visualize and derive insights from models' output

A human-machine interface (HMI) is an interface that enables humans to monitor and control a device. An HMI is required in a control system and is usually connected to a single PLC or process to enable operators to change the flow set point and enable alarm conditions in the event of a loss of flow, high temperature, or anomalies, and the condition is displayed and recorded. The following figure shows a customized HMI connected to Ultron that streams the AI model’s output, visualizes important device metrics, and includes interactive elements for I/O controls

Edge Management

Deploying and managing a large network of AI infrastructure at the edge remains a challenge for organizations especially when the edge devices are spread across geographically dispersed locations. From remote debugging, software management and maintenance to system monitoring, a centralized management suite like FleetTrackr provides a simplified and secured method to provision, manage, maintain, monitor, and update thousands of appliances, entirely over-the-air. For example, Ultron could be widely distributed across vast distances when deployed for industrial automation and FleetTrackr can be used to streamline management.

There are six key components in FleetTrackr provisioned by the FleetTrackr Management Suite:

• Order Management: Keep track of order status and manage device inventory

• Device Management: Easily monitor device metrics and device status, get access to crucial device performance KPIs based on historical data. Access device documents, specification sheets, and user manuals for ease of deployment.

• Container Management: Update or restart your container with a single click and create a group of devices to provision containers easily.

• Firmware-Over-The-Air: Firmware update, backup and recovery through remote access

• Issue Management: Group edge devices and raise tickets when a device goes faulty, create and manage site leaders and teams to resolve tickets, schedule regular device maintenance tasks, and get access to KPIs on ticket history.

• Predictive Maintenance: Detect hardware/software anomalies through automated anomaly detection and classification, predict time-to-failure

In short, FleetTrackr is readily available to be used for deploying and managing Ultron at scale. Instead of spending weeks planning and executing deployment plans, administrators can upgrade AI solutions, manage applications, update system firmware and software, streamline operations and administrative tasks, and monitor the health metrics of the fleets from a single management panel.

Use Cases

INDUSTRIAL AUTOMATION FOR SMARTER
FACTORIES

To date, a large portion of industrial automation is based on logic-based control from PLCs for a variety of processes and systems. The fundamental role of a PLC is to automate processes by sending programmed control functions computed from signals received from connected inputs such as a sensor, switch, thermometer, or relay to output devices. This system architecture, also known as a supervisory control and data acquisition (SCADA) system, is a secure and standardized automation infrastructure for sharing diagnostics and analytics. For example, PLCs are often used in industrial machine vision and automation systems for image processing and quality inspection, in pick-and-place operation of autonomous forklifts and robotic arms in manufacturing environments, as well as in downstream to upstream process control in oil and gas plants.

PLC implementation in action

PLCs have traditionally been widely used due to their dependability and robustness, as they require minimal intervention to operate. Optical quality inspection using machine vision systems is one of the most widely used methods for ensuring quality control in high-precision manufacturing. In production, for example, machine vision systems use sensors (cameras), processing hardware and software algorithms to automate complex or mundane optical inspection tasks and precisely guide handling equipment during product assembly.

Bringing deep learning to the factory floor

With the emergence of Industrial Internet-of-Thing (IIoT), Artificial Intelligence-of-Thing (AIoT) and robotics, manufacturers are developing smart factory strategies that use AI on automated manufacturing processes in production environments, freeing up people to focus on handling exceptions and making higher-level decisions in order to remain competitive in the industry. For example, to achieve higher detection accuracy, manufacturers are beginning to use deep learning technologies and real-time computer vision applications that can be performed at the edge. Traditional inspection tasks use rules-based machine vision solutions for part inspection and leak detection, but deep learning approaches such as convolutional neural networks (CNNs) can help to optimize inspection accuracy without any impact on production.

Ultron for smarter factories

To address this, SmartCow designed an all-in-one control platform that combines a wide range of industrial functionalities such as machine vision, PLC, AIoT, and robotics, complemented by a control software. This control software integrates I/O control, computer vision, and video analytics by providing the necessary vision and PLC capabilities in the form of function libraries, I/O blocks, and APIs that can be called up from Ultron, all configurable through a browser-based programming tool, allowing engineers to program in modern programming languages such as JavaScript and Python. Furthermore, NVIDIA hardware-accelerated SDKs such as the Isaac SDK for robotics algorithm development and the Deepstream SDK for streaming video analytics can be used to create high-performance AI applications. In terms of connectivity, the dual ethernet ports, 5G/LTE, Wifi, CANBus, and I2C options make Ultron an ideal PLC for multiple protocols and communication support.

Conclusion

To summarize, Ultron is built to improve industrial automation processes that lack the agility, programmer productivity, and development scalability needed for effectively interacting with sophisticated next-generation AI computing workloads that are often deployed on the cloud. We discovered that advanced computing capabilities are in high demand at the industrial edge control system. Nonetheless, SmartCow's approach of using Jetson-based programmable logic controllers to easily integrate edge AI with flexible industrial automation technology is applicable to a wide range of industries beyond factory automation, and it is the company's goal to innovate next generation solutions toward industry 4.0.

SMART TRAFFIC: SIGNAL CONTROL AT INTERSECTIONS

With the launch of CityStation, an AIoT platform for environmental data acquisition and monitoring, SmartCow is one step closer to boosting smart cities initiatives with weather data-driven services. While our team was working on a Jetson-based PLC for industrial automation, we discovered the potential of Ultron's to be deployed at traffic intersections alongside CityStation for smart traffic signal control.

Building a smart traffic signal control module

A traffic signal box has two main components: a controller and a conflict monitor. The controller within the traffic signal box serves as the traffic signal's brain. Depending on the intersection's phases (4 or 8 phases), it takes input from proximity sensors and loop detectors installed beneath the road or from video streams from birds' eye view cameras overseeing the intersection, which uses pixelation changes to estimate traffic flows. After that, the controller is programmed with vehicle movements, lanes, and timing sequences to control traffic flow and adjust traffic signals as needed. The conflict monitor, on the other hand, serves as the traffic signal control system's security guard. When it detects malfunction signals in conflicting directions, it shuts down the signals and returns to blinking red / yellow. When a conflict is detected, it exchanges data packets with the controller and turns the signal lights at the intersection on flush. It also monitors the controller's power voltage.

The problem with conventional traffic signal control

Although this conventional method of calculating traffic signal timing is more reliable and less prone to error, it does have some drawbacks. While the timing controller fixes the time sequence for every phase, it does not adapt to the intersection's real-time traffic. For example, the road most vehicles use to get to work in the morning may be more congested in the morning but less traveled in the afternoon, and it is usually the opposite direction that is congested. Furthermore, the installation of proximity sensors and loop detectors beneath the road is costly and resource intensive.

Ultron for adaptive traffic signal control

To solve this problem, SmartCow's AI engineering team proposes using Ultron as an adaptive traffic light controller based on computer vision AI. The signal system adapts to the current traffic situation by performing inference on live feed from a fusion of AI sensors and machine vision cameras installed at traffic junctions to calculate real-time traffic density by detecting and classifying vehicles at the intersections. The vehicle detection model calculates traffic density at each phase, and the algorithm sends the scheduling timing to the controller along with the optimized green and red signal duration. Aside from traffic density, the model considers the number of lanes on the road, vehicle types, and buffer time for vehicles to restart after stopping. A conflict monitor is also added to the PLC module that ensures that road safety is prioritized. Urban engineers can also view data using SmartCow's RoadMaster, an AI-enabled traffic management system for real-time data analysis that enables capabilities such as traffic anomaly and violation detection, turn counts, and lane utilization analysis.

An adaptive traffic signal management system not only reduces traffic congestion on the road, but it also contributes to lower CO2 emissions and shorter travel times for road users. Furthermore, adaptive traffic signal control relies less on loop detectors and proximity sensors that are often expensive and not robust to withstand harsh weather conditions.

SMART HOME: HOME AUTOMATION AND ACCESS CONTROL

A smart home can be achieved by using a home automation system that integrates smart technologies such as IoT, sensors, electrical appliances, and mobile devices, allowing homeowners to monitor and control their home. A smart home project is easy to begin with. In fact, many homeowners began their home automation project by purchasing a small single-board computer, such as the Raspberry Pi, and open source programming tools. Another option for a control module for smarthome systems is a PLC, which is commonly used for controlling electromechanical processes. There is no end to the scenarios that a smart home can achieve.

Here are some examples:

• Access controlled doors, gates, and blinds

• Remote and intelligent control of air conditioning systems

• Adjustment of lighting intensity based on user preferences

• Smart light switching in the mornings and at night

• Heating water bathing and making tea

• Temperature and pressure control in a swimming pool or jacuzzi

• Controls for pump motors used in garden irrigation and water fountains

• Suspicious activity detection in the home, which automatically activates the alarm system and sends alert messages

Ultron, which is based on NVIDIA® Jetson™platforms, is an excellent choice for the aforementioned use cases. It includes control software that is simple to use and requires minimal programming knowledge or integration with other software. Whether or not you are familiar with NVIDIA DeepStream SDK, a streaming analytics toolkit for AI-based multi-sensor processing video or image, is an excellent tool for AI application development. NVIDIA DeepStream SDK supports multiple frameworks and extensive reference of pretrained models. The DeepStream toolkit is readily available for use with Ultron's I/O modules, allowing you to apply programmable logic based on sensor readings and video inference results and send appropriate output signals to control the devices.