The Picky Pigeons¶

This section of the course handbook has been provided as part of the final group project for the Seminar on Tracking Animal Behavior at the Ruhr-University Bochum in WS21/22.

Project Authors

Jil, MSc Psychology, Lea, M.Sc. Psychology, Laurin, M.Sc. Psychology, and Maurice, M.Sc. Psychology.

Abstract¶

Analyzing animal behavior has always brought up certain challenges, most of which can be overcome using the open source, markerless source estimation toolbox DeepLabCut. When performing a DeepLabCut analysis the most important decision is the number of tracking points used for training the model.

To investigate how this influences the softwares confidence, we trained two models using the same dataframes but different numbers of tracking points. We used top-view videos of eight pigeons in a cage and extracted 20 frames per video for labeling. For the less complicated model we used seven tracking points; for the more complicated model we added an extra of 15 points, resulting in overall 22 tracking points. We hypothesise the following:

There is a lower pixel difference in the complex model.
The confidence of the model is higher in the complex model.

Additionally, we investigated how the different complexity levels influence the work flow of labeling the frames. Thus, each group member filled out the Flow-Kurz Skale (Rheinberg, 2000) while labeling each model. We hypothesise that:

The experienced flow is higher in the easy model (measured with the Flow-Kurzskala (FKS).

Overall, we try to figure out, what number of tracking points yields the best balance between workload and accuracy.

Fig. 19 Sample of test and training frames from deeplabcut.evaluate_network()¶

Data Analysis¶

1. pixel difference/confidence of model:¶

First, we describe descriptively the mean pixel errors in both models.
The simple model resulted for 1030000 training iterations with a p-cutoff of 0.6 in a test error of 6.7 pixels whereas the complex model resulted in a test error of 6.5 pixels. That means, the complex model was in average 0.2 pixels better in predicting the points. Although it may be not statistically significant there seems to be a tendency of higher confidence and lower mean pixel error in the complex model than in the easy model. This is contrary to our proposed hypothesis.

2. flow:¶

Here, we describe our results of our questionnaire to experienced flow.
In the easy model the average flow experience was 50 (range: 7-70), while in the complex model it was 53.25. Besides the flow experience we assessed the concern about the exercise via the flow-scale, resulting in an average of 10 (range: 3-30) in the easy model and an average of 11 in the complex model.
That said, contrary to our hypothesis, the flow seemed to be higher in labeling the complex model instead of the easy model.

3. explorative: Position in space¶

We chose the backcenter point as the point to track the position of the pigeon in space. This point is enclosed in both models which enables a comparison of the models. For comparison, we plot the positions in a scatter plot.

First, import packages and define file, how to read file, and relevant information of file.

# import relevant packages
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

# define file that is used, how and what to read of the csv file, and define which columns are used in coords
file = 'C:/Users/USER/Documents/Picky_Pigeon_Project-The_Picky_Pigeons-2021-12-17/videos/166_filtered - Kopie.csv'
data = pd.read_csv(file, header=0) #headings of the columns are = row 0
coords = data.loc[:,data.columns.str.contains('backcenter')] #all rows and all columns that contain backcenter define coords

# output coords
coords

	backcenter	backcenter.1	backcenter.2
0	819.974548	494.949646	0.999818
1	819.925218	494.904734	0.999708
2	821.008876	495.086856	0.999528
3	822.399030	495.153400	0.999729
4	823.219302	494.142786	0.999571
...	...	...	...
71920	564.234147	517.139613	0.999623
71921	564.201800	517.161179	0.999610
71922	564.292883	517.262576	0.999495
71923	564.375360	517.289639	0.999680
71924	564.437141	517.552503	0.999595

71925 rows × 3 columns

Here, define your first plot of position over time.

# define your plot with the defined coords backcenter as x values and backcenter.1 as y values and label the plot
position_x = coords['backcenter']
position_y = coords['backcenter.1']

pos_x, = plt.plot(position_x, label='x')
pos_y, = plt.plot(position_y, label='y')

plt.xlabel('Time [frames]')
plt.ylabel('Position [mm]')
plt.legend()

<matplotlib.legend.Legend at 0x197ae19b760>

This is how it looks like in the complex model:

from IPython.display import Image
Image("F:/Picky_Pigeon_Project-The_Picky_Pigeons_part2-2021-12-17/PlotComplex.png")

You can also define histogram plots for the x and y values each:

# histogram plot with x values
position_x.hist()

<AxesSubplot:>

# histogram plot with y values
position_y.hist()

<AxesSubplot:>

This is how it looks like in the complex model:

from IPython.display import Image
Image("F:/Picky_Pigeon_Project-The_Picky_Pigeons_part2-2021-12-17/histComplex.png")

And finally also a scatter plot:

# create a 2d plot
x = coords['backcenter']
y = coords['backcenter.1']

fig = plt.scatter(x,y,marker='o', s = 10, c='green');

# adding title and labels
plt.title("2D-model of backcenter in space in the easy model")
plt.xlabel("X-axis")
plt.ylabel("Y-axis")

# displaying plot
plt.show()

This is how it looks like in the complex model:

from IPython.display import Image
Image("F:/Picky_Pigeon_Project-The_Picky_Pigeons_part2-2021-12-17/complexScatter.png")

Conclusion¶

Based on our data analysis the complex and simple model seemed not to differ significantly in accuracy of tracking. It may be that a model with more tracking points is better regarding prediction error and flow experience in a bigger study with more data. However, there may be non-linearic correlations between number of tracking points and accuracy of the model and between number of tracking points and flow-experience. There is no evidence yet and this research question is left for future research.

Tracking Animal Behavior

The Picky Pigeons

Contents

The Picky Pigeons¶

Abstract¶

Data Analysis¶

1. pixel difference/confidence of model:¶

2. flow:¶

3. explorative: Position in space¶

Conclusion¶

Notes¶

For starting Deeplabcut use one of the following code cells¶

Trouble Shooting¶

here are some errors and solutions that might occur while creating your project:¶

This is our own project - named Picky Pigeons¶

DeepLabCut Notebook¶

Starting an existing project¶

step 1: import python libraries¶

step 2: open existing project using config_path command¶

For labeling frames use the following code chunks:¶

step 3: Label frames¶

step 4: Check frames¶

For creating the training data set and training the model use the code chunks of steps 5 and 6:¶

For creating a new DeepLabCut Project use the following code blocks¶

Configure Project¶

These are the body parts and skeleton connections we used for the easy model:¶

These are the bodyparts and skeleton configs we used for the complex model:¶

Extract frames from the videos¶

Label Frames¶

Step 5: create training data set:¶

Alternative 1¶

Alternative 2¶

Alternative 1¶

Step 6: Train Model¶

Evaluate Model¶

Analyze Videos¶

Create labeled videos¶

Load Existing Model¶

Analyze new videos with the previously trained model¶

Extract outlier frames¶

Merge Datasets¶

Re-Train Network¶

Start over again…¶