Compressed-Sensing Recovery of Images and Video
Using Multihypothesis Predictions

Chen Chen, Eric W. Tramel, and James E. Fowler

Abstract

Compressed-sensing reconstruction of still images and video sequences driven by multihypothesis predictions is considered. Specifically, for still images, multiple predictions drawn for an image block are made from spatially surrounding blocks within an initial non-predicted reconstruction. For video, multihypothesis predictions of the current frame are generated from one or more previously reconstructed reference frames. In each case, the predictions are used to generate a residual in the domain of the compressed-sensing random projections. This residual being typically more compressible than the original signal leads to improved reconstruction quality. To appropriately weight the hypothesis predictions, a Tikhonov regularization to an ill-posed least-squares optimization is proposed. Experimental results demonstrate that the proposed reconstructions outperform alternative strategies not employing multihypothesis predictions.

Download

[PDF] [MATLAB CODE] [PRESENTATION]

Experiment Results

1. Video
The performance of multihypothesis predictions and residual reconstruction is evaluated on 4 different video sequences. More results can be found at http://www.ece.msstate.edu/~ewt16/publications/

Table 1. PSNR in dB of Residual Reconstruction (RR) of Video

Algorithm

Subrate

0.1 0.2 0.3 0.4 0.5

Foreman

RR w/ MH-TIK31.533.835.035.936.7
RR w/ MH-l128.031.032.934.536.3
BCS-SPL25.127.529.931.833.7

Susie

RR w/ MH-TIK34.236.337.338.038.6
RR w/ MH-l130.733.435.236.738.1
BCS-SPL29.431.933.635.136.4

Football

RR w/ MH-TIK25.527.629.130.331.4
RR w/ MH-l124.626.728.329.731.1
BCS-SPL24.025.927.228.529.9

News

RR w/ MH-TIK30.931.632.032.132.2
RR w/ MH-l123.727.029.531.634.1
BCS-SPL20.223.426.328.931.8

2. Still Images
The performance of the MH-BCS-SPL and MH-MS-BCS-SPL is evaluated on 13 grayscale images of size 512 x 512. More details about BCS-SPL algorithm can be found at http://www.ece.msstate.edu/~fowler/BCSSPL/

Table 2. Image Reconstruction PSNR (dB) at Various Subrates

Algorithm

Subrate

0.1 0.2 0.3 0.4 0.5

Lenna

BCS-SPL27.9331.2733.4135.1536.71
MH-BCS-SPL29.8532.8534.7336.3437.82
MS-BCS-SPL31.4834.5936.6137.8238.94
MH-MS-BCS-SPL31.6134.8836.7938.3239.74
MS-GPSR30.3033.6035.2936.2937.73
TV29.8432.9335.0536.8238.43

Barbara

BCS-SPL22.2823.7825.3226.8828.51
MH-BCS-SPL27.8931.4633.6335.6837.29
MS-BCS-SPL23.8425.1226.0527.2728.83
MH-MS-BCS-SPL24.2826.4227.9832.9536.21
MS-GPSR24.0325.2426.0627.5029.64
TV22.9524.4826.3028.4230.78

Goldhill

BCS-SPL26.5728.9230.4231.7233.03
MH-BCS-SPL27.6730.2831.8233.2634.62
MS-BCS-SPL28.9731.0632.7533.6734.64
MH-MS-BCS-SPL29.0731.3533.0634.5536.10
MS-GPSR28.3930.5532.1032.8934.20
TV27.5229.8731.6333.2334.81

Mandrill

BCS-SPL20.4521.8022.9023.9625.10
MH-BCS-SPL20.4722.3624.0325.3626.67
MS-BCS-SPL21.4523.0724.6925.5426.46
MH-MS-BCS-SPL21.6623.2024.8225.8127.10
MS-GPSR21.5222.9224.2725.0726.21
TV20.5322.0223.4424.9426.52

Peppers

BCS-SPL28.8331.9833.7335.1236.36
MH-BCS-SPL30.2832.8234.3235.6336.87
MS-BCS-SPL31.7534.5935.7636.7737.67
MH-MS-BCS-SPL32.0834.7335.9637.1538.37
MS-GPSR29.2531.9033.0334.2235.75
TV30.3633.1434.7035.9137.05

Clown

BCS-SPL24.4629.6531.8733.7035.37
MH-BCS-SPL28.7832.8035.0236.8338.40
MS-BCS-SPL29.0632.7235.6536.9438.02
MH-MS-BCS-SPL29.4833.8936.2038.1639.76
MS-GPSR27.9331.5833.9535.0036.56
TV27.9431.1833.4835.4137.24

Crowd

BCS-SPL24.4928.8731.1633.1134.98
MH-BCS-SPL26.6230.1132.3534.3836.22
MS-BCS-SPL29.1532.7235.7137.0338.32
MH-MS-BCS-SPL29.2032.7635.7837.5739.33
MS-GPSR27.9931.2034.1335.1436.96
TV26.8130.4533.2635.8138.32

Couple

BCS-SPL24.7827.0128.6530.1431.60
MH-BCS-SPL25.8228.8530.6732.2133.76
MS-BCS-SPL26.8729.5131.4732.5533.65
MH-MS-BCS-SPL27.0430.0531.7933.8335.57
MS-GPSR26.2828.8230.6031.6133.15
TV25.7328.4830.6432.6134.51

Man

BCS-SPL26.2528.6330.3031.7933.24
MH-BCS-SPL26.7529.3630.9532.4734.03
MS-BCS-SPL28.6431.0533.0334.0535.10
MH-MS-BCS-SPL28.6831.2233.1834.5936.12
MS-GPSR27.7830.1531.8533.0134.51
TV27.2529.9432.0533.9835.98

Boat

BCS-SPL25.1427.7829.5731.1232.60
MH-BCS-SPL26.1729.3031.1832.8934.45
MS-BCS-SPL27.3530.0831.9833.1234.22
MH-MS-BCS-SPL27.4630.3832.2333.8935.46
MS-GPSR26.5229.2130.8432.0633.64
TV26.4229.2431.2832.9734.53

Cameraman

BCS-SPL26.0230.3633.6536.4238.90
MH-BCS-SPL29.8633.9736.5739.2841.48
MS-BCS-SPL31.0536.5439.9842.9144.91
MH-MS-BCS-SPL31.7138.0842.8545.6348.15
MS-GPSR29.5334.7639.5941.8245.08
TV30.8735.0838.3241.0943.73

Girl

BCS-SPL29.6332.6934.7636.4638.07
MH-BCS-SPL31.7434.8436.6838.4139.85
MS-BCS-SPL33.4936.2938.8739.7040.63
MH-MS-BCS-SPL33.5836.7539.0840.4341.81
MS-GPSR32.1835.2437.7038.5939.89
TV30.2633.5335.8737.7739.51

Barbara2

BCS-SPL23.6425.5627.1928.7530.38
MH-BCS-SPL26.8330.1932.0333.7135.27
MS-BCS-SPL25.1427.3129.1230.3431.66
MH-MS-BCS-SPL25.6128.8731.0633.7436.30
MS-GPSR25.2827.3428.8230.1932.21
TV23.9026.2428.5130.7432.97

Visual Comparison

The following example shows a visual comparison of the various algorithms. All the images are reconstructed using different algorithms at sampling subrate = 0.1.

Lenna

BCS-SPL Reconstructed Image, PSNR = 28.03 dB MH-BCS-SPL Reconstructed Image, PSNR = 29.76 dB
MS-BCS-SPL Reconstructed Image, PSNR = 31.43 dB MH-MS-BCS-SPL Reconstructed Image, PSNR = 31.55 dB
MS-GPSR Reconstructed Image, PSNR = 30.28 dB TV Reconstructed Image, PSNR = 29.83 dB

Barbara

BCS-SPL Reconstructed Image, PSNR = 22.35 dB MH-BCS-SPL Reconstructed Image, PSNR = 27.91 dB
MS-BCS-SPL Reconstructed Image, PSNR = 23.91 dB MH-MS-BCS-SPL Reconstructed Image, PSNR = 24.31 dB
MS-GPSR Reconstructed Image, PSNR = 24.03 dB TV Reconstructed Image, PSNR = 22.95 dB

Cameraman

BCS-SPL Reconstructed Image, PSNR = 26.17 dB MH-BCS-SPL Reconstructed Image, PSNR = 29.77 dB
MS-BCS-SPL Reconstructed Image, PSNR = 31.01 dB MH-MS-BCS-SPL Reconstructed Image, PSNR = 31.75 dB
MS-GPSR Reconstructed Image, PSNR = 29.52 dB TV Reconstructed Image, PSNR = 30.77 dB