…is designed to deliver solutions for Smart Multimedia Information Retrieval for highly efficient, effective, scalable, integrable, and explainable applications.
The SMIRA Platform is based on the developments in the context of Smart Multimedia Information Retrieval located at the University of Hagen and various other R&D organizations. Particularly, the Generic Multimedia Analysis Framework (GMAF), the Multimedia Feature Graph (MMFG), Graph Codes, and MMIR Explainability are part of the core functions of SMIRA.
SMIRA is a integrable and standard-compatible platform, which provides numerous import- and export facilities als well as an enhanced set of MMIR feature extraction plugins (open source and commercial). The configuration of SMIRA is based on processing flows.
Processing Flows are defined from source to target and contribute MMIR features to a MMIR collection. Within such a flow, various components can be arranged to provide the requested MMIR processing. This includes feature extraction, currently with the following list of major extractors:
- Image
- EXIF Data including Camera / Lens model and Location
- Object Detection Plugins by
- Google Vision
- Microsoft Vision
- Amazon AWS
- ClarifAi
- Carnet (Car and Car type specialist)
- Open CV
- Yolo Fruit
- Yolo General
- Social Media (Facebook, Instagram, Twitter)
- Text Annotations
- Video
- FFMPEG for Video-Processing and Conversion (all known formats)
- Scene Detection and Splitting
- MPEG7 and MXF Containers
- Keyframe Extraction
- Audio Track Extraction
- Google Vision Video Summary
- Live Streaming and Video on Demand
- Social Media (Facebook, Instagram, Twitter)
- Audio
- Speech to Text
- Shazam Integration for Artist, Title, Label
- Beats Per Minute Extractor
- Pitch Extractor
- Text
- Microsoft Office (including Word, Excel, and embedded Multimedia Objects)
- Bag Of Words Algorithm
- TFIDF Calculation
- Sentence and topic detection
- RSS Extractors
- XML and XSLT
- RTF
- Other
- SQL/JDBC integration
- XML
Based on these extractors and the processing flows into the collection, further export flows can be automated. These can be exported via File, Database, or Stream into common Media Formats (e.g., Word, Excel, SQL, Image, etc.), but also including some common standard formats, like MPEG7, MXF, MOS, BXF, XML.
Technically, SMIRA is based on the Generic Multimedia Analysis Framework (GMAF).
For further information, also see the R&D section here.