Blurry-Edges Dataset

Back to project main page Download dataset

Overview

This page provides an introduction to the Blurry-Edges dataset, detailing the training, validation, testing sets, and real-captured data along with their purpose, structure, and usage. First, the optics setup is listed below.

Parameter	Value	Unit
Pixel pitch	5.86e-6	m
Sensor distance	0.1104	m
Optical power #1	10.0	dpt
Optical power #2	10.2	dpt
Aperture diameter	0.006	m
Working range	0.75-1.18	m

All data can be downloaded via OneDrive. The training, validation, and testing data are stored in .npy format with .png previews provided in .zip files for visualization. The dataset is organized in a hierarchical folder structure. Click on the folder icons to expand/collapse the contents:

- Blurry-Edges dataset Root dataset folder
- + data_train_val Training and validation sets (basic geometrics; 147 by 147 image size)
  - images_ny_train.npy 8,000 pairs of noisy images for global stage training
  - images_ny_val.npy 2,000 pairs of noisy images for global stage validation
  - images_gt_train.npy 8,000 pairs of clean images for global stage training
  - images_gt_val.npy 2,000 pairs of clean images for global stage validation
  - derivative_maps_train.npy 8,000 pairs of derivative maps for global stage training
  - derivative_maps_val.npy 2,000 pairs of derivative maps for global stage validation
  - images_aif_train.npy 8,000 AIF images for global stage training
  - images_aif_val.npy 2,000 AIF images for global stage validation
  - alphas_train.npy 8,000 alpha values for global stage training
  - alphas_val.npy 2,000 alpha values for global stage validation
  - boundary_locations_train.npy 8,000 boundary location maps for global stage training
  - boundary_locations_val.npy 2,000 boundary location maps for global stage validation
  - boundary_distances_train.npy 8,000 boundary distance maps for global stage training
  - boundary_distances_val.npy 2,000 boundary distance maps for global stage validation
  - boundary_depths_train.npy 8,000 boundary depth maps for global stage training
  - boundary_depths_val.npy 2,000 boundary depth maps for global stage validation
  - image_depths_train.npy 8,000 depth maps for global stage training
  - image_depths_val.npy 2,000 depth maps for global stage validation
  - + patches Cropped patch samples (21 by 21 patch size)
    - patches_ny_train.npy 16,000 noisy patches for local stage training
    - patches_ny_val.npy 4,000 noisy patches for local stage validation
    - patches_gt_train.npy 16,000 clean patches for local stage training
    - patches_gt_val.npy 4,000 clean patches for local stage validation
    - derivative_maps_train.npy 16,000 derivative map patches for local stage training
    - derivative_maps_val.npy 4,000 derivative map patches for local stage validation
    - patches_aif_train.npy 16,000 AIF patches for local stage training
    - patches_aif_val.npy 4,000 AIF patches for local stage validation
    - alphas_train.npy 16,000 alpha values for local stage training
    - alphas_val.npy 4,000 alpha values for local stage validation
    - boundary_locations_train.npy 16,000 boundary location map patches for local stage training
    - boundary_locations_val.npy 4,000 boundary location map patches for local stage validation
    - boundary_distances_train.npy 16,000 boundary distance map patches for local stage training
    - boundary_distances_val.npy 4,000 boundary distance map patches for local stage validation
    - boundary_depths_train.npy 16,000 boundary depth map patches for local stage training
    - boundary_depths_val.npy 4,000 boundary depth map patches for local stage validation
    - image_depths_train.npy 16,000 depth map patches for local stage training
    - image_depths_val.npy 4,000 depth map patches for local stage validation
    - + train_preview Local stage training data visualization
      - xxx.zip multiple .zip files
    - + val_preview Local stage validation data visualization
      - xxx.zip multiple .zip files
  - + train_preview Global stage training data visualization
    - xxx.zip multiple .zip files
  - + val_preview Global stage validation data visualization
    - xxx.zip multiple .zip files
- + data_test Testing set (synthetic)
  - + regular Testing data used for evaluation (147 by 147 image size)
    - images_ny.npy 200 pairs of noisy images for testing
    - alphas.npy 200 alpha values for testing
    - depth_maps.npy 200 depth maps for testing
    - images_ny_preview.zip visualization for noisy images
    - depth_maps_preview.zip visualization for depth maps
  - + large Testing data used for exploration (659 by 659 image size)
    - images_ny.npy 10 pairs of noisy images for exploration
    - alphas.npy 10 alpha values for exploration
    - depth_maps.npy 10 depth maps for exploration
    - images_ny_preview.zip visualization for noisy images
    - depth_maps_preview.zip visualization for depth maps
- + data_real_captured Real-captured data (real data; 5 scenes and 2 exposure time values; 299 by 299 image size)
  - + exposure_time_4395.25us Data captured with 4395.25us exposure time
    - xxx Multiple folders for different scenes
  - + exposure_time_10011.37us Data captured with 10011.37us exposure time
    - xxx Multiple folders for different scenes

Training and validation sets

The training and validation sets are located in ./data_train_val folder, containing 8,000 and 2,000 randomly generated samples, respectively. They consist solely of basic geometrics, i.e., rectangles, circles, and triangles. We provide a pair of defocused noisy images, a pair of defocused clean images, a pair of derivative maps, an all-in-focus image, a photo level value, a boundary map, a boundary distance map, a boundary depth map, and a depth map for each sample. The image size is 147 by 147. To accelarate the local stage training, we randomly cropped 16,000 and 4,000 patch samples from the training and the validation sets, respectively, focusing on regions with prominent boundaries. The patch size is 21 by 21. These patch samples are stored in ./data_train_val/patches folder. Both sets can be generated using train_val_data_generator.py script available in the GitHub repository.

Sample of full images

Noisy image #1

Noisy image #2

Clean image #1

Clean image #2

Derivative map #1

Derivative map #2

All-in-focus image

Photon level

Boundary map

Boundary distance map

Boundary depth map

Depth map

Sample of patches

Noisy image

Clean image

Derivative map

All-in-focus image

Photon level

Boundary map

Boundary distance map

Boundary depth map

Depth map

Testing set

Testing set is located in ./data_test folder. The background is randomly selected from the Painting dataset, and the foreground uses images from the MS-COCO dataset. There are two versions, regular and large image sizes. The regular version contains 200 samples with 147 by 147 image size, while the large version contains 10 samples with 659 by 659 image size. Each sample includes a pair of defocused noisy images, a photon level value, and a depth map. The data can be generated using test_data_generator.py script available in the GitHub repository.

Sample with regular image size

Noisy image #1

Noisy image #2

Photon level

Depth map

Sample with large image size

Noisy image #1

Noisy image #2

Photon level

Depth map

Real-captured data

Real-captured data is located in ./data_real_captured folder. It contains 5 scenes captured with two different exposure times, 4395.25us and 10011.37us, to demonstrate the versatility of the Blurry-Edges approach. Each scene includes a stack of defocused noisy images (on this page, we only show a pair) and a depth map. The image size is 299 by 299. The setup we used is shown in the next section.

Sample

Noisy image #1

Noisy image #2

Depth map

Camera setup

Using a deformable lens

In the paper and dataset, we used a prototype camera system that is similar to the setup in Focal Track, including an Optotune EL-16-40TC-VIS-5D-C to change the optical power and a FLIR Grasshopper3 GS3-U3-23S6C-C camera.

Using a beamsplitter

Alternatively, we provide another setup with a beamsplitter and a fixed lens. In this setup, we use two Basler daA3840-45uc cameras with different sensor distances. This allows for more flexibility in adjusting the focus and depth of field.