Challenge Results
Thank you all for participanting the challenge. Congratulations to challenge winners:
Track 2
- 1st Place: Fujitsu Research
- 2nd Place: Kathy Wu (Amazon MENA Tech) and Sarthak Srivastava (Amazon Global)
- 3rd Place: Alin-Ionut Popa, George Leotescu, Daniel Voinea (Amazon Trustworthy Shopping Experience)
| Participants | AUC CMC (Top-30) | AUC CMC (Top-5) | mAP |
|---|---|---|---|
| Fujitsu Research | 96.307 | 78.929 | 96.955 |
| Kathy Wu (Amazon MENA Tech) and Sarthak Srivastava (Amazon Global) | 57.552 | 32.254 | 28.810 |
| Alin-Ionut Popa, George Leotescu, Daniel Voinea (Amazon Trustworthy Shopping Experience) | 54.753 | 28.983 | 23.739 |
Physical Retail Challenges
As part of this workshop, three challenges will be opened using the introduced real-world physical retail data collected in grocery store environments, named Grocery Vision 2023. This dataset contains individual shopping actions of anonymized customers collected with a GoPro camera mounted on a standard US shopping cart. Three challenges are:
- Track 1: Video Temporal Action Localization (TAL) and Spatial Temporal Action Localization (STAL):
TAL and STAL challenges aims at localizing the actions of interest in the video frames. For TAL, the output of the model is the temporal segments of the actions of interest. For STAL, the output of the model is the bounding boxes localizing the actions of the interest. In this task, we focus on the products and actions that associate with potential shopping behaviors, i.e. products that is taken into the basket, and products that is taken out of the basket. To evaluate model's performance, frame-mAP and tube-mAP will be used.
In this task, participants will be provided with 73,683 images and annotation pairs for download as training set.
- Track 2: Appearance Based Verification (ABV):
ABV challenge aims at validating if the model is able to match the query product image to the corresponding image in gallery. Participants will be given a training dataset consisting of images for different products and are expected to design methods for accurate matching product identities. During testing, held out query images will be used to probe the proposed model. The model is expected to output a ranking list of items based on the matching the model predicts. To evaluate model's performance, Cumulative Matching Characteristics (CMC) curves will be used.
In this task, participants will be provided with 74,200 images as training set. Each image file name contains the barcode of the corresponding product which is considered as the ID for matching.
Download the dataset:
This dataset is released under CC BY-NC 4.0 license, please review the license, and send us an email to weijianl@amazon.com to download the dataset with your: (1) name, (2) email (affliation email, not @gmail etc.), (3) purpose of using the dataset. We will response with the download link as soon as possible.
Submit results:
To participate in the evaluation, please submit the files necessary to run inference with your method. It will be runned on held out testsets to benchmark with the other submissions.