REVERSE VIDEO SEARCH API FUNCTIONAL DOCUMENTATION
The TECXIPIO Reverse Video Search API allows for searching for highly altered versions, near-duplicates, and duplicates of reference videos that the user has added to their storage.
Each time the user conducts a search request, they provide a video to be checked against this storage.
Please click here if you are looking for the technical API documentation.
DEFINITIONS
VIDEO FINGERPRINTING
All video content that is added to the storage, as well as videos that are uploaded for search requests, is fingerprinted first. For standard API usage, this fingerprinting step is transparent to the user.
In any case, the matching process is not based on the original video files, but the digital fingerprints of the videos stored in our back-end. In addition, when conducting a search request, digital fingerprints of the search video are generated and checked against the database. The transmitted video files are not stored in persistent storage at any time and are removed from RAM when the video fingerprints are generated. Furthermore, video frames cannot be reconstructed from the fingerprints.
Optional Video Fingerprinting Tool
For confidential video content, it is possible to provide a video fingerprinting tool that allows the reference videos to be fingerprinted on the user’s premises so that only the fingerprints of the reference videos are transferred to our servers.
Throughout the rest of this document, the term "video" is used to describe the content, even though only the fingerprints of these are processed.
COLLECTIONS
The storage is split up into "collections", which can be understood as folders in the back-end– an entity that allows the reference videos of a user to be organized.
Multiple Collections
If you always want to check your search videos against all videos in your reference storage in the future, it is sufficient to add all reference videos to a single collection. Otherwise, organizing the reference videos into several collections is recommended, depending on the need. Let us say you have videos of specific topics and want to search only in one topic, you should create collections for topic 1 to x to save money and time.
Adding/Removing Collections
Collections can be added or removed via our API. For each video registration or search, you need to provide the collection for which the operation should be performed.
Throughout the rest of this document, the term "storage" is used for the set of all user collections.
SEARCH TYPES
During search time, the user has the choice between two search types: standard and advanced. Note that this choice does not already have to be made when adding reference videos to the storage, but is a flexible choice for each search request.
Cost of Search Types
In contrast to standard searches, an advanced search is considerably more robust to partial occlusions, sub-imaging, and super-imaging, which comes at additional costs. The costs for standard and advanced searches can be compared with the aid of our cost calculator, which is provided on our homepage. Click here to calculate the costs of video fingerprinting and matching with the different search types.
Standard vs. advanced search type
The standard search is robust to the following:
-
Rotation
-
Scaling
-
Modified brightness
-
Slight tolerance regarding partial occlusion, sub-frames, and super-frames
The advanced search is robust to the following:
-
Rotation
-
Scaling
-
Modified brightness
-
Partial occlusions
-
Sub-frames
-
Super-frames
Initially testing with both search types is recommended for highly altered videos to determine if the advanced search is actually required for the use case at hand. If not, costs can be saved by using the standard search.
ADDITIONAL TRANSFORMATION TYPES
Independent from the search types mentioned above, several additional transformation types are available for video fingerprinting and searching. Please note, that additional transformation types must be activated separately by contacting us.
The additional transformation types are as follows:
-
Mirroring
-
Stretching
-
Squeezing
-
Mirroring + Stretching
-
Mirroring + Squeezing
In contrast to standard or advanced searches, the additional transformation types can only be included during search time if the reference videos were previously added to the storage accordingly, since additional video fingerprints are needed.
In other words, matching a search video against a reference video with a specific additional transformation type requires reference videos to have been added, including that transformation type. However, the opposite method is flexible: one can perform searches with only one transformation type, for example, even when the videos were added to the storage for all available transformation types.
These additional transformation types can be understood as transformations that are applied to the reference video already when computing its fingerprint(s). Each additional transformation results in an additional video fingerprint, i.e. without any additional transformation types, only one fingerprint (the one for the original video) is added. With all five transformation types, six fingerprints (including the one for the original video) are added to the storage in total. When searching for a specific transformation type, the algorithm tries to find matches only for videos that were added to the storage with this transformation type. Videos without this transformation type are not compared and do not produce any costs.
The adding and search costs, therefore, increase in accordance with the additional number of video fingerprints. For instance, adding a reference video with all five additional transformation types produces six times the cost compared with adding the reference video without any additional transformation types. The same applies to searches. However, you are only billed according to what is actually compared. When all videos are added with the transformation types “original” and “mirroring”, searching with the transformation type “stretched” will not compare any videos or produce costs. Searches with “original” or “mirroring” will perform a comparison of all the added videos in the storage.
MATCH RESULT
A match result consists of the following information (the full table can only be viewed on desktop):
matches
match:
video_id
video_match
video_match:
coverage
avg_features_matches
std_features_matches max_features_matches
duration_quest
duration_match
frame_matches
frame_match:
feature_matches
features_quest
features_match
timestamp_quest
timestamp_match
[]match
uint32
video_match
float32
float32
float32
uint32
float32
float32
[]frame_match
uint32
uint32
uint32
float32
float32
ID of the matched reference video
The video match information
A number between 0 and 1 (percent/100) estimating the percentage of the reference video covered by the match
Average number of feature matches
Standard deviation of feature matches
Maximum number of feature matches
Duration of search video in seconds
Duration of matched reference video in seconds
The set of frame matches
Number of feature matches
Number of features in the search video frame
Number of features in the matched reference frame
Timestamp of the search video frame, in seconds
Timestamp of the matched reference frame, in seconds
A feature is a local feature computed by a keypoint detector.
The match confidence of a keyframe match is beyond 99% according to our statistical evaluation and is therefore not contained in frame_match.
The coverage estimate is computed based on the timestamp information of the matched keyframes.
USE CASES
REGISTER
To perform searches, you initially need to add videos to a collection in your storage. You need your collection ID (found in the TECXIPIO App) and the video file you want to add. Currently, we support MP4, Matroska, WebM, and AVI. Codec H.264.
Registering Videos
You can register videos with a POST request to /collections/[COLLECTION_ID]/videos/.
A video file is expected by default.
Uploading Pre-calculated Video Fingerprints
With the query parameter format=fingerprint you can upload pre-calculated fingerprints that were generated with our optional Video Fingerprinting Tool.
Search- and Transformation Types
If the transformation type parameter is not set, only the "original" type is calculated. You can specify the transformation type with its query parameter transformation-types=[BITMASK].
We support original: 1, flipped-x: 2, stretched: 4, squeezed: 8, flipped-x stretched:16, and flipped-x squeezed: 32.
If you were going to create the original and flipped-x type for the uploaded video, the bitmask parameter would be 3.
As result, you will obtain the ID of the video that has been added to your storage.
Status of Registering Task
Since the processing of an uploaded video can take some time, the process is asynchronous. With a GET request to /collections/[COLLECTION_ID]/videos/[VIDEO_ID]/status the status of the register task can be requested.
Add and Fetch Meta Information
You can even add meta information to your stored video. This includes the title, year, and a comment, which can be added with a PUT request to /collections/[COLLECTION_ID]/videos/[VIDEO_ID]/details.
With a GET call to the same URL, you can fetch existing video details.
With registered videos, you are prepared to perform searches with our Reverse Video Search API.
SEARCH
Searches run on specific collections. The video for the search must be in one of our supported formats: MP4, Matroska, WebM or AVI. Codec H.264.
Enqueue searches
Searches must be enqueued at first. Send your request with a POST call to /collections/[COLLECTION_ID]/adhoc-searches/enqueue.
Search- and Transformation Types
For the usage of the "format" and "transformation-types" query parameters, please check the Use Case: Register. If you are searching for a specific transformation type, matching is only performed for the reference videos that were uploaded with this transformation type. In addition, you can use the query parameter "search-type" to specify whether you want to use the standard or advanced search engine.
Status of search task
Check the status of a search task with a GET request to /collections/[COLLECTION_ID]/adhoc-searches/[SEARCH_ID]/status.
Fetch results
After a search is finished, you can fetch the results with a GET request to /collections/[COLLECTION_ID]/adhoc-searches/[SEARCH_ID].