REVERSE VIDEO SEARCH API FUNCTIONAL DOCUMENTATION

The TECXIPIO Reverse Video Search API allows for searching for highly altered versions, near-duplicates, and duplicates of reference videos that the user has added to their storage.

Each time the user conducts a search request, they provide a video to be checked against this storage.

Please click here if you are looking for the technical API documentation. 

USE CASES

 

Register

Search

DEFINITIONS

VIDEO FINGERPRINTING

All video content that is added to the storage, as well as videos that are uploaded for search requests, is fingerprinted first. For standard API usage, this fingerprinting step is transparent to the user.

In any case, the matching process is not based on the original video files, but the digital fingerprints of the videos stored in our back-end. In addition, when conducting a search request, digital fingerprints of the search video are generated and checked against the database. The transmitted video files are not stored in persistent storage at any time and are removed from RAM when the video fingerprints are generated. Furthermore, video frames cannot be reconstructed from the fingerprints.

Optional Video Fingerprinting Tool

For confidential video content, it is possible to provide a video fingerprinting tool that allows the reference videos to be fingerprinted on the user’s premises so that only the fingerprints of the reference videos are transferred to our servers.

Throughout the rest of this document, the term "video" is used to describe the content, even though only the fingerprints of these are processed.

COLLECTIONS

The storage is split up into "collections", which can be understood as folders in the back-end– an entity that allows the reference videos of a user to be organized.

Multiple Collections

If you always want to check your search videos against all videos in your reference storage in the future, it is sufficient to add all reference videos to a single collection. Otherwise, organizing the reference videos into several collections is recommended, depending on the need. Let us say you have videos of specific topics and want to search only in one topic, you should create collections for topic 1 to x to save money and time.

Adding/Removing Collections

Collections can be added or removed via our API. For each video registration or search, you need to provide the collection for which the operation should be performed.

Throughout the rest of this document, the term "storage" is used for the set of all user collections.

SEARCH TYPES

During search time, the user has the choice between two search types: standard and advanced. Note that this choice does not already have to be made when adding reference videos to the storage, but is a flexible choice for each search request.

Cost of Search Types

In contrast to standard searches, an advanced search is considerably more robust to partial occlusions, sub-imaging, and super-imaging, which comes at additional costs. The costs for standard and advanced searches can be compared with the aid of our cost calculator, which is provided on our homepage. Click here to calculate the costs of video fingerprinting and matching with the different search types. 

Standard vs. advanced search type

The standard search is robust to the following:

  • Rotation

  • Scaling

  • Modified brightness

  • Slight tolerance regarding partial occlusion, sub-frames, and super-frames

The advanced search is robust to the following:

  • Rotation

  • Scaling

  • Modified brightness

  • Partial occlusions

  • Sub-frames

  • Super-frames

Initially testing with both search types is recommended for highly altered videos to determine if the advanced search is actually required for the use case at hand. If not, costs can be saved by using the standard search.
 

ADDITIONAL TRANSFORMATION TYPES

Independent from the search types mentioned above, several additional transformation types are available for video fingerprinting and searching. Please note, that additional transformation types must be activated separately by contacting us.

 

The additional transformation types are as follows:

  • Mirroring

  • Stretching

  • Squeezing

  • Mirroring + Stretching

  • Mirroring + Squeezing

In contrast to standard or advanced searches, the additional transformation types can only be included during search time if the reference videos were previously added to the storage accordingly, since additional video fingerprints are needed.

In other words, matching a search video against a reference video with a specific additional transformation type requires reference videos to have been added, including that transformation type. However, the opposite method is flexible: one can perform searches with only one transformation type, for example, even when the videos were added to the storage for all available transformation types. 


These additional transformation types can be understood as transformations that are applied to the reference video already when computing its fingerprint(s). Each additional transformation results in an additional video fingerprint, i.e. without any additional transformation types, only one fingerprint (the one for the original video) is added. With all five transformation types, six fingerprints (including the one for the original video) are added to the storage in total. When searching for a specific transformation type, the algorithm tries to find matches only for videos that were added to the storage with this transformation type. Videos without this transformation type are not compared and do not produce any costs.

The adding and search costs, therefore, increase in accordance with the additional number of video fingerprints. For instance, adding a reference video with all five additional transformation types produces six times the cost compared with adding the reference video without any additional transformation types. The same applies to searches. However, you are only billed according to what is actually compared. When all videos are added with the transformation types “original” and “mirroring”, searching with the transformation type “stretched” will not compare any videos or produce costs. Searches with “original” or “mirroring” will perform a comparison of all the added videos in the storage.
 

MATCH RESULT

A match result consists of the following information (the full table can only be viewed on desktop):

matches 

      

match:

video_id     
video_match 

video_match:

coverage                          

 

avg_features_matches 

std_features_matches max_features_matches 

duration_quest     

duration_match   

frame_matches 

frame_match:
feature_matches  
features_quest   
features_match      
timestamp_quest    
timestamp_match 
 

[]match

 

 

uint32

video_match 

 

 

float32

float32 

float32

uint32

float32

float32

[]frame_match 

 

uint32

uint32

uint32

float32

float32 

ID of the matched reference video
The video match information

 

A number between 0 and 1 (percent/100)                estimating the percentage of the reference video covered by the match

Average number of feature matches

Standard deviation of feature matches

Maximum number of feature matches

Duration of search video in seconds

Duration of matched reference video in seconds

The set of frame matches

Number of feature matches

Number of features in the search video frame

Number of features in the matched reference frame

Timestamp of the search video frame, in seconds

Timestamp of the matched reference frame, in        seconds
 

A feature is a local feature computed by a keypoint detector.

The match confidence of a keyframe match is beyond 99% according to our statistical evaluation and is therefore not contained in frame_match.

The coverage estimate is computed based on the timestamp information of the matched keyframes. 

 
 
 
 
 
 

USE CASES

REGISTER

To perform searches, you initially need to add videos to a collection in your storage. You need your collection ID (found in the TECXIPIO App) and the video file you want to add. Currently, we support MP4, Matroska, WebM, and AVI. Codec H.264.

Registering Videos

You can register videos with a POST request to /collections/[COLLECTION_ID]/videos/.

A video file is expected by default.

Uploading Pre-calculated Video Fingerprints

With the query parameter format=fingerprint you can upload pre-calculated fingerprints that were generated with our optional Video Fingerprinting Tool

 

Search- and Transformation Types

If the transformation type parameter is not set, only the "original" type is calculated. You can specify the transformation type with its query parameter transformation-types=[BITMASK].

 

We support original: 1, flipped-x: 2, stretched: 4, squeezed: 8, flipped-x stretched:16, and flipped-x squeezed: 32.

If you were going to create the original and flipped-x type for the uploaded video, the bitmask parameter would be 3.

As result, you will obtain the ID of the video that has been added to your storage.

Status of Registering Task

Since the processing of an uploaded video can take some time, the process is asynchronous. With a GET request to /collections/[COLLECTION_ID]/videos/[VIDEO_ID]/status the status of the register task can be requested.

Add and Fetch Meta Information

You can even add meta information to your stored video. This includes the title, year, and a comment, which can be added with a PUT request to /collections/[COLLECTION_ID]/videos/[VIDEO_ID]/details.

With a GET call to the same URL, you can fetch existing video details.

With registered videos, you are prepared to perform searches with our Reverse Video Search API.

SEARCH

Searches run on specific collections. The video for the search must be in one of our supported formats: MP4, Matroska, WebM or AVI. Codec H.264.

Enqueue searches

Searches must be enqueued at first. Send your request with a POST call to /collections/[COLLECTION_ID]/adhoc-searches/enqueue.

 

Search- and Transformation Types

For the usage of the "format" and "transformation-types" query parameters, please check the Use Case: Register. If you are searching for a specific transformation type, matching is only performed for the reference videos that were uploaded with this transformation type. In addition, you can use the query parameter "search-type" to specify whether you want to use the standard or advanced search engine.

Status of search task

Check the status of a search task with a GET request to /collections/[COLLECTION_ID]/adhoc-searches/[SEARCH_ID]/status.

Fetch results

After a search is finished, you can fetch the results with a GET request to /collections/[COLLECTION_ID]/adhoc-searches/[SEARCH_ID].


 

 
 

Contact us: 

info@tecxipio.com

REVERSE VIDEO SEARCH

ABOUT US

German flag - link to German domain

GENERAL

© TECXIPIO GmbH

TECXIPIO_Logo_Claim_RGB.png
TECXIPIO icon