Exploring the UCF 101 Action Dataset
Generate image frames from video
From the UCF 101 dataset website: UCF101 is an action recognition data set of realistic action videos, collected from YouTube, having 101 action categories. We downloaded a zipped file containing 13320 trimmed videos, each including one action, and a text file containing the list of action classes and their numerical index.
Unzip the .avi files into a VIDEO_DIR folder.
import glob
video_files = glob.glob(os.path.join(os.environ["VIDEO_DIR"], "*.avi"))
video_files[0:2]
There is better support for .mp4 files, so we will convert the .avi files to .mp4 using ffmpeg. The code below requires that your machine have ffmpeg installed.
import subprocess
def convert_from_avi_to_mp4(file_name):
outputfile = file_name.lower().replace(".avi", ".mp4")
subprocess.call(['ffmpeg', '-i', file_name, outputfile])
The code below takes quite a while and could be sped up by using multi-processing:
for file_name in video_files:
convert_from_avi_to_mp4(file_name)
Once all the files are converted we can list all the .mp4 files
mp4_video_files = glob.glob(os.path.join(os.environ["VIDEO_DIR"], "*.mp4"))
mp4_video_files[0:2]
Select one of the videos at random for illustration:
from IPython.display import Video
from random import choice, seed
seed(457)
video_path = choice(mp4_video_files)
video_path
Display the selected video:
Video(video_path, embed=True)
A video is a sequence of image frames. In this section we want to create a function that generates image frames at a given rate of frames per second and then select a specific number of equally spaced image frames to be kept for further processing.
import numpy as np
def sample_images(images, number_to_sample):
"""
Sample equally spaced frames from a list of image frames
:param images: a list of image frames.
:param number_to_sample: int representing the number os frames to sample.
:return: a numpy array containing the sample of frames.
"""
if number_to_sample < len(images):
idx = np.round(np.linspace(0, len(images) - 1, number_to_sample)).astype(int)
return np.array(images)[idx]
else:
return np.array(images)
import imageio
def extract_images(video_path, number_frames):
"""
Extract equally spaced frames from a video.
:param video_path: Full .mp4 video path.
:param number_frames: Number of frames to sample.
:return: a numpy array containing the sample of frames.
"""
reader = imageio.get_reader(
video_path,
fps=1
)
frames = []
for i, im in enumerate(reader):
frames.append(im)
return sample_images(frames, number_frames)
Generate 4 equally spaced frames from the video displayed above.
images = extract_images(video_path, 4)
Plot the extracted frames:
import matplotlib.pyplot as plt
def plot_images(images):
fig = plt.figure(figsize=(10, 10))
for idx, image in enumerate(images):
sub = fig.add_subplot(2,2,idx+1)
imgplot = plt.imshow(image)
plt.tight_layout()
plot_images(images)
