Query-aware adaptive windowing for spatiotemporal complex video event processing for internet of multimedia things
MetadataShow full item record
This item's downloads: 476 (view details)
With the evolution of the Internet of Things (IoT), there is an exponential rise in sensor devices that are deployed ubiquitously. Due to the extensive usage of IoT applications in smart cities, smart homes, self-driving cars, and social media, there is enormous growth in multimedia data streams like videos and images. We are now transitioning to an era of the Internet of Multimedia Things (IoMT), where unstructured data like videos are continuously streamed from visual sensors like CCTV cameras and smartphones. Video data is highly expressive and has traditionally been very difficult for a machine to interpret. Middleware systems such as Complex Event Processing (CEP) mine patterns from data streams and send notifications to users in a timely fashion. Current CEP systems have inherent limitations to mine event patterns from video streams due to their unstructured data model, lack of expressive query language, and resource intensiveness. This work introduces VidCEP, a data-driven, distributed, on the fly, near real-time complex event matching framework for video streams with five key contributions: 1) Expressive Video Event Query Language- Current event query languages are highly focused on temporal reasoning. A SQL-like declarative Video Event Query Language (VEQL) is proposed which enables state-based spatiotemporal video event matching without needing to focus on low-level video features. The VEQL enables the creation of robust spatiotemporal operators using a hybrid approach. 2) Structured Video Stream Representation- The work introduces Video Event Knowledge Graph (VEKG), a knowledge graph driven model of video data streams using an ensemble of deep learning models and VEQL operators. VEKG creates a semantic knowledge representation of video data by modeling video objects as nodes and their relationship interaction as edges over time and space. 3) Query and State-aware stream summarization- Objects coexist across multiple frames, leading to the creation of redundant nodes and edges at different time instances that result in high memory usage and increased matching time. The work introduces two novel state and multi-query based spatiotemporal graph summarizations of VEKG streams- Time Aggregated Graph (VEKG-TAG) and Event Aggregated Graph (VEKG-EAG) for an efficient state-based event matching. 4) Resource constraint distributed content-driven windowing- An adaptive and content-driven windowing technique VID-WIN is proposed to improve Quality of Service (QoS). VID-WIN instances are deployed over edge and cloud to perform efficient state-based video event matching. The VID-WIN adopts resource and query-aware runtime optimization strategies to improve the CEP matching performance under limited available resources and application requirements. 5) VidCEP Complex Event Processing Framework: The above techniques are integrated into Video Complex Event Processing framework (VidCEP). A prototype of the framework is developed which performs query-based video event pattern matching in a distributed setting. The VidCEP capability is demonstrated using a real-work traffic estimation service for OpenStreetMap. A total of 18 event rules are defined from the traffic management and activity recognition domain. Extensive experiments have been performed across 20 datasets consisting of more than 3900 video clips to evaluate the performance and efficacy of proposed techniques. Results of this study show that VidCEP achieves a throughput of approximately 70 frames per second (fps) for five parallel streams (at 17fps) with sub-second matching latency. The system successfully detects different spatiotemporal video event patterns with good F-scores (0.51-0.90). The optimization techniques proposed in the thesis improves ~5X search time, ~2.3X throughput, and ~99% bandwidth savings.
Except where otherwise noted, this item's license is described as Attribution-NonCommercial-NoDerivs 3.0 Ireland
Showing items related by title, author, creator and subject.
The Irish National Adverse Events Study (INAES): the frequency and nature of adverse events in Irish hospitals—a retrospective record review study Rafter, Natasha; Hickey, Anne; Conroy, Ronan M; Condell, Sarah; O'Connor, Paul; Vaughan, David; Walsh, Gillian; Williams, David J (BMJ, 2016-02-09)Introduction Irish healthcare has undergone extensive change recently with spending cuts and a focus on quality initiatives; however, little is known about adverse event occurrence. Objective To assess the frequency and ...
Preconditioning shields against vascular events in surgery (saves), a multicentre feasibility trial of preconditioning against adverse events in major vascular surgery: study protocol for a randomised control trial Healy, Donagh; Clarke-Moloney, Mary; Gaughan, Brendan; O’Daly, Siobhan; Hausenloy, Derek; Sharif, Faisal; Newell, John; O’Donnell, Martin; Grace, Pierce; Forbes, John F; Cullen, Walter; Kavanagh, Eamon; Burke, Paul; Cross, Simon; Dowdall, Joseph; McMonagle, Morgan; Fulton, Greg; Manning, Brian J; Kheirelseid, Elrasheid AH; Leahy, Austin; Moneley, Daragh; Naughton, Peter; Boyle, Emily; McHugh, Seamus; Madhaven, Prakash; O’Neill, Sean; Martin, Zenia; Courtney, Donal; Tubassam, Muhammed; Sultan, Sherif; McCartan, Damian; Medani, Mekki; Walsh, Stewart (Springer Nature, 2015-04-23)Background: Patients undergoing vascular surgery procedures constitute a 'high-risk' group. Fatal and disabling perioperative complications are common. Complications arise via multiple aetiological pathways. This mechanistic ...
Knowledge graph driven approach to represent video streams for spatiotemporal event pattern matching in complex event processing Yadav, Piyush; Salwala, Dhaval; Das, Dibya Prakash; Curry, Edward (World Scientific Publishing, 2020)Complex Event Processing (CEP) is an event processing paradigm to perform real-time analytics over streaming data and match high-level event patterns. Presently, CEP is limited to process structured data stream. Video ...