Efficient and Adaptive Computer Vision for Cyber-physical Systems

Lan, Shuyue

doi:https://doi.org/10.21985/n2-7qr4-sa78

Work

Efficient and Adaptive Computer Vision for Cyber-physical Systems

Public

Download PDF

Download All Files (.zip)

With the rapid advancement of machine learning techniques (in particular deep neural networks), computer vision applications have shown great promises in a variety of domains for intelligent cyber-physical systems (CPSs), such as autonomous driving, medical imaging, and vision-based robotic systems. However, while many vision applications provide great on-paper performance, their system realizations still face significant challenges and often fall short of the expectations in practice. In particular, vision-based intelligent systems often operate in a dynamic and uncertain environment, with changing mission requirements, and based on limited resources of computation, communication, data storage and energy availability. Moreover, the vision algorithms themselves are usually computationally intensive and resource hungry. This presents tremendous challenges in ensuring the system performance, efficiency, safety, robustness and predictability in operation. These challenges promote the development of methodologies, algorithms and tools for building real-world efficient and adaptive vision-based intelligent systems. In this dissertation, to improve the efficiency of vision-based CPSs, we start from the single-agent adaptation design. On the one hand, we hope to select the minimal set of data that should be analyzed to meet the scene understanding objective. Hence,we develop an efficient online video fast-forwarding approach (FFNet) based on reinforcement learning, which can automatically fast-forward a video and present a representative subset of frames to users on the fly. It does not require processing the entire video, but just the portion that is selected by the fast-forward agent, which makes it very computationally efficient. On the other hand, to handle the various factors of vision application environment, a framework that can adaptively select the algorithm-parameter combinations according to the changing physical environment, mission requirements, and resource constraints is developed, with the application of pedestrian detection. We then consider multi-agent applications, which have recently gained immense popularity. In many computer vision tasks, a network of agents, such as a team of robots with cameras, could work collaboratively to perceive the environment for efficient and accurate situation awareness. However, these agents often have limited computation, communication, and storage resources. To address these challenges in multi-agent systems, we develop a distributed framework DMVF and a centralized framework MFFNet for multi-agent video fast-forwarding. Both methods can fast-forward multi-view video streams collaboratively and adaptively. We also consider energy-harvesting, intermittently-powered sensors that have emerged as a zero maintenance solution for long-term environmental perception. These devices suffer from intermittent and varying energy supply. Therefore, we further develop an adaptive environment monitoring framework, AdaSens, which adapts the operations of intermittently-powered sensor nodes in a coordinated manner to cover as much as possible of the targeted scene.

Creator