Amazon Picking Challenge Object Scans

Robot Learning Lab, UC Berkeley

Arjun Singh, Karthik Narayan, Ben Kehoe, Sachin Patil, Ken Goldberg, Pieter Abbeel

bigbird@lists.eecs.berkeley.edu

Here you can find data we have collected for the objects used in the Amazon Picking Challenge. The data has been collected and processed using the same system described in the ICRA 2014 publication A Large-Scale 3D Database of Object Instances and the ICRA 2015 publication Range Sensor and Silhouette Fusion for High-Quality 3D Scanning.
Specifically, for each object, we provide:

Note that some objects, depending on their properties (e.g. transparency) may not have complete point clouds or meshes. We include them because the raw data and/or the partial meshes may still be useful. Specifically, first_years_take_and_toss_straw_cups, munchkin_white_hot_duck_bath_toy, and safety_works_safety_glasses have significantly below-average quality models.

You can access the data via this Google Drive link.

To immediately get started loading Kinbodies into OpenRAVE, you can use the files in 'kinbody.tgz' for each object.

Available Data

For each object, we provide three types of .tgz files. The data contained in each is described below.

OpenRAVE Kinbody Files (object_name/kinbody.tgz)

The "Kinbody" file (object_name/kinbody.tgz) contains:
  • Reconstructed mesh models (e.g. "meshes/poisson.stl")
  • Kinbody files for using the meshes with OpenRAVE (e.g. "kinbody/poisson.kinbody.xml")
  • A recommended Kinbody for file for each object ("kinbody/recommended.kinbody.xml"). This file is the one we felt was best after visually inspecting each model.

Raw RGB-D (object_name/rgbd.tgz)

The "Raw RGB-D" file (object_name/rgbd.tgz) contains:
  • For each of 5 RGB-D sensors and 120 turntable positions (600 total):
    • raw depth maps (e.g. "NP2_60.h5", in 100um)
    • raw RGB images (e.g. "NP2_60.jpg") from the Primesense sensors
    • segmentation masks for the RGB images (e.g. "masks/NP2_60.pbm"); generated from object models, currently missing for objects for which models could not be obtained
    • turntable pose information for each image (e.g. "poses/NP5_60_pose.h5")
  • calibration information for each RGB-D sensor (RGB intrinsic matrix, IR/depth intrinsic matrix) ("calibration.h5")

Raw High Resolution RGB (object_name/highres.tgz)

The "Raw RGB" file (object_name/highres.tgz) contains:
  • For each of 5 high-resolution RGB cameras and 120 turntable positions (600 total):
    • raw RGB images (e.g. "N2_60.jpg") from the Canon cameras
    • segmentation masks for the images (e.g. "masks/N2_60.pbm"); generated from object models, currently missing for objects for which models could not be obtained
    • turntable pose information for each image (e.g. "poses/NP5_60_pose.h5")
  • calibration information for each RGB camera (RGB intrinsic matrix) ("calibration.h5")

Processed Data (object_name/processed.tgz)

The "Processed" file (object_name/processed.tgz) contains:
  • Reconstructed mesh models (e.g. "meshes/poisson.ply")
  • Textured mesh models (e.g. "textured_meshes/poisson.obj")
  • Detected chessboard corners for the turntable chessboard in the reference camera's images (e.g. "corners/NP5_60_corners.txt")
  • For each of 5 high-resolution RGB cameras and 120 turntable positions (600 total):
    • turntable pose information for each segmented cloud (e.g. "poses/NP5_60_pose.h5")
  • calibration information for each RGB camera (RGB intrinsic matrix) ("calibration.h5")

Naming

The RGB-D sensors have names of the form "NP[1-5]." The RGB cameras have names of the form "N[1-5]." The cameras are arranged in a quarter-circular arc, and the overhead position is labeled 5, and the lowest-to-the-ground position is labeled 1. The names are shown in the image below. These names are used everywhere, including filenames and in calibration information.

Calibration and Pose Information

The calibration file contains the intrinsic matrix for each Canon RGB camera, and for the RGB and IR/depth cameras for each Primesense Carmine.

One of the overhead cameras is designated as the "reference camera." Currently, the reference camera for all objects is NP5. Relative transformations (given as homogeneous transformation matrices) between all cameras and the reference camera are provided in each 'calibration.h5' file. Note that different objects may have different calibration information. Using the relative transforms, one may transform points from any camera's frame into any other camera's frame.

The pose files (e.g. "poses/NP5_60_pose.h5") contain the transformation matrix from the reference camera to the turntable frame for each turntable position.

By combining the calibration relative transformations with the reference-to-table transformations, one can obtain pose information for every image or point cloud.

Download

You can access the data via this Google Drive link.

Image Name Kinbodies Raw RGB-D Raw high resolution Processed
champion_copper_plus_spark_plug Kinbodies (.tgz) RGB-D (.tgz) High res (.tgz) Processed (.tgz)
cheezit_big_original Kinbodies (.tgz) RGB-D (.tgz) High res (.tgz) Processed (.tgz)
crayola_64_ct Kinbodies (.tgz) RGB-D (.tgz) High res (.tgz) Processed (.tgz)
dove_beauty_bar Kinbodies (.tgz) RGB-D (.tgz) High res (.tgz) Processed (.tgz)
dr_browns_bottle_brush Kinbodies (.tgz) RGB-D (.tgz) High res (.tgz) Processed (.tgz)
elmers_washable_no_run_school_glue Kinbodies (.tgz) RGB-D (.tgz) High res (.tgz) Processed (.tgz)
expo_dry_erase_board_eraser Kinbodies (.tgz) RGB-D (.tgz) High res (.tgz) Processed (.tgz)
feline_greenies_dental_treats Kinbodies (.tgz) RGB-D (.tgz) High res (.tgz) Processed (.tgz)
first_years_take_and_toss_straw_cups Kinbodies (.tgz) RGB-D (.tgz) High res (.tgz) Processed (.tgz)
genuine_joe_plastic_stir_sticks Kinbodies (.tgz) RGB-D (.tgz) High res (.tgz) Processed (.tgz)
highland_6539_self_stick_notes Kinbodies (.tgz) RGB-D (.tgz) High res (.tgz) Processed (.tgz)
kong_air_dog_squeakair_tennis_ball Kinbodies (.tgz) RGB-D (.tgz) High res (.tgz) Processed (.tgz)
kong_duck_dog_toy Kinbodies (.tgz) RGB-D (.tgz) High res (.tgz) Processed (.tgz)
kong_sitting_frog_dog_toy Kinbodies (.tgz) RGB-D (.tgz) High res (.tgz) Processed (.tgz)
kygen_squeakin_eggs_plush_puppies Kinbodies (.tgz) RGB-D (.tgz) High res (.tgz) Processed (.tgz)
laugh_out_loud_joke_book Kinbodies (.tgz) RGB-D (.tgz) High res (.tgz) Processed (.tgz)
mark_twain_huckleberry_finn Kinbodies (.tgz) RGB-D (.tgz) High res (.tgz) Processed (.tgz)
mead_index_cards Kinbodies (.tgz) RGB-D (.tgz) High res (.tgz) Processed (.tgz)
mommys_helper_outlet_plugs Kinbodies (.tgz) RGB-D (.tgz) High res (.tgz) Processed (.tgz)
munchkin_white_hot_duck_bath_toy Kinbodies (.tgz) RGB-D (.tgz) High res (.tgz) Processed (.tgz)
one_with_nature_soap_dead_sea_mud Kinbodies (.tgz) RGB-D (.tgz) High res (.tgz) Processed (.tgz)
oreo_mega_stuf Kinbodies (.tgz) RGB-D (.tgz) High res (.tgz) Processed (.tgz)
paper_mate_12_count_mirado_black_warrior Kinbodies (.tgz) RGB-D (.tgz) High res (.tgz) Processed (.tgz)
rollodex_mesh_collection_jumbo_pencil_cup Kinbodies (.tgz) RGB-D (.tgz) High res (.tgz) Processed (.tgz)
safety_works_safety_glasses Kinbodies (.tgz) RGB-D (.tgz) High res (.tgz) Processed (.tgz)
sharpie_accent_tank_style_highlighters Kinbodies (.tgz) RGB-D (.tgz) High res (.tgz) Processed (.tgz)
stanley_66_052 Kinbodies (.tgz) RGB-D (.tgz) High res (.tgz) Processed (.tgz)