Introduction

In this article will write down what is going on during the coding period and keep it updated with the work done every week and what is expected to be done at each week and keep it up to date with any new updates with the organization and the mentors guidance (frequently updated)

Updating note

Due to some personal reasons I have used the extended gsoc projects timeline to fit my current availability so the blogs gets updated bi-weekly instead of weekly. so by the end of the session I will have completed the same standard's weeks count within the extended timeline;

The project requirements are to be established with the project's mentors through holding a meeting discussing what is supposed to be done through the current summer.

After holding a meeting with the mentors we have make an initial state to start from the last year's version so we have made a quick overview on the proposal and get which parts are important to be done through this year and which ones we can get it as a low prioritized missions

What is going on

This section will be a set of lists each list contains the accomplished task within a specific week

Week no. 01

Reviewing some parts on the project
Notification to the experiment owner when clustering job is done through mail
Making the co-ordinates viewed in the view mode of the experiment
Allowing the experiment owner to disable or enable the annotation with co-ordinates(image)/captions(audio/video/text)
System design for the new part of adding the classification inference part

Week no. 02

Hold a meeting with all mentors attending to make an establishment to what will be done through this year
Making some configurations to the new added features last year (whether you need it switch on/off)
- Labels' others text field for each selected label became configurable through annotation level settings
Going to transform the functional requirements into engineering requirements and discuss that with mentors

Week no. 03

Established the requirements of the project with the mentors and going to finalize that and make a fully perspective of the final output what should it be.
Finished the experiment owner's own space annotation levels sharing, which allows the experiment owners to import their annotation levels from another experiments without making them globally to all of the other users.

Week no. 04

Added sharing experiment's annotation levels to certain users feature

Week no. 05

I have been working on the ELAN functionalities integration to the Rapid annotator ELAN, the required currently is adding the time-stamp annotations to videos with multiple tracks/tiers selection. I have started exploring what a starting point I can start from and continue and after a while I have suggested a base start code from a timeline-video project called videotimeline a demo of that can be accessed from here it dose not contain all of required functionalities / most of our need is missed but as starting point looks interesting as it use web-graphics solution (canvas) and after looking surround the available software solutions for video editing that provides a timeline for the videos (most of them has been using a graphics library for building their UI and the timeline) in the mean while our need is looking forward an accurate frame accessing for the videos to select the exact time. As a conclusion what is needed is adding a new experiment type, going to name it elan and that experiment is accepting videos/audios types and for each experiment it will have (tracks/tiers) like the annotations levels but with another name, every annotation will be attached to a (track/tier/annotation level) but this time each video file will have un-limited annotations entry as it is like manual transcription. In the following demo it has an expectation of the output (from elan tool)

Week no. 06

Currently I am working on the customization of timeline.js tools to create the needed functionalities of ELAN experiment type inside the rapid annotator.

Current supported functionalities

Loading text and writing it on the track/tier (annotation text)
Deleting a specific annotation
Zoom in/out (through context-menu and mouse-wheel)
Scroll down/up (through mouse-wheel and dragging scroll-bars)
Navigation through video
Fixed navigation synchronization with the video and timeline time-ticker
Fixed elements (tracks/tiers) overlapping with the time scale element and the buttons as well as fixed the overlapping of the annotation with the time-ticks elements/lines

TODO functionalities

Editing text/annotation on tracks/tiers (currently working on)
Adding new annotations/text dynamically on the tracks/tiers (currently working on)
Adding frame navigation to the current navigation using VideoFrame.js
Range selection (start and end time) for the annotation selection
Dragging and dropping the annotations (not urgent may be delayed after integration)
Exporting the timeline/elan
Integration into rapid annotator

Live preview of what is done (frequently gets update) here!

Week no. 07

Currently added supported functions

Add annotation
Edit Annotation
Remove Annotation
Annotation selection on top
Multiple mouse wheel modes (zoom or scroll on mouse wheel event)
Time validation and annotation form validation
Start/End time selection from ticks timeline (context menu)

Currently working on

Load preset annotations options
Frame and seconds seeker/stepper
Exporting the timeline/elan
Integration into rapid annotator

Demo of what is currently done

The live preview is kept up to date here!

Week 08 & 09 & 10

Finally finished the ELAN type annotation experiment with the required functionatlies From Rapid annotator to ELAN software exported file

Firstly when creating a new experiment, you will need to select it as elan type

Moving into the experiment page same as the other type of experiments
Upload your videos normally
Add your Annotation levels but this time they are named as "Tiers"

Add labels for each tier

Add annotators for the experiment

Start Annotation

Add Annotation
Added Annotation
Edit Annotation

Edited Annotation

Demos
- Adding annotations
- Update annotations

- Discard/Delete annotations

Week 11 & 12

Added Elan labels shortcut keys added and fixed an issue; fixed the labels' data-list after adding the numbers shortcut keys for the tiers' selection (the labels options were not being changed)

Enable/Disable advanced options in the experiment and fixed specific view A new configuration to the experiment to control whether you want to add the new features those added last year in that PR Annotations adds-on (coordinates feature) #60 so, at the adding new experiment we get a new field for the experiment's table called advanced annotation and at the creation of the experiment it appears as a checkbox.
At viewing a specific entry of the experiment files (the advanced features above were missing from that view, so they have been added and are being loaded and edited also from specific annotation view)

After	Before

PR Demos

1 - Checkbox for each annotation level to configure whether you want to allow the text-field for each selected text box or not through the annotation level settings from the experiment's labels page

In case of making it on Text fields next to the selected label

In case of making it off No text fields next to the selected label

labelconfig2

2 - Allowing the experiments' owners to import their annotation levels without doing the over-head step [make it global, import it then make it private again] users-own-levels-import

3 - Allowing sharing the annotation levels of an experiment to certain users selection from the settings page

Screenshot (Experiment settings page) 2share-levels

4- Elan experiment

Description Elan experiment: is a new category of experiments (extension to video experiment) that is needed to be added to rapid annotator that allows multiple and infinite count of annotations for each level. New analogies (for elan experiment):
Level: Tier
Label: (Annotation) Text

The structure of each annotation entry will be as the following (description of database changes): Note: Database schema updated image is added in docs folder ElanAnnotation:

id - Int (auto increment)
experimentId - the experiment is being in the process
fileId - the file is being annotated
data - json ; that is the point to be explained

Review of what is the replacement: each selected label was recorded as a single record in AnnotationInfo table. For elan it is instead of keeping a single record for each entry (as they are infinite; not limited to the count of labels inside each level) it was better to use a json type So far, data is a json type contains the following:

{
"levelId01": [{"startTime": 0.01, "endTime": 0.036, "text": "Anythingcan be here"}]
}

It is an object contains (levels/tiers ids of the experiment as keys and for each tierId it contains an array which is the content of annotations) Sample view explained that image is represneted as the following:

{
  "34": [
    {
      "text": "label 1",
      "endTime": 6.5,
      "startTime": 0
    },
    {
      "text": "label 3",
      "endTime": 19.25,
      "startTime": 10.0625
    }
  ],
  "35": [
    {
      "text": "dsa",
      "endTime": 15,
      "startTime": 2.4375
    }
  ]

Results is exported in 2 ways

All results of the experiment at once in a tabular format contains the following columns (['File Name', 'Annotator', 'Tier Name', 'Annotation Text', 'Start Time (sec)', 'End Time (sec)']) as (csv, xlsx)
Export a .eaf (elan file format) of a specific user's annotation for a specific file
User interface changes: *
When creating a new experiment, you will find a new type

At view results page you will find a new column that allows you to download a .eaf file for each annotation of the selected user and clicked file

New experiment interface (elan-similar)

Key bindings

Add/Edit annotation form
Add/Edit annotation form Demo

Timeline
Timeline Demo

Functions

Zoom in/out
Scroll (zoom mode or scroll up/down mode)
Hide other annotations of a track when on is selected, or show all
Control video buttons

Complete demo (video) for annotation to export results! Video on Youtube

5- Elan labels shortcut keys added and fixed an issue #67

Added the shortcut key of each label below the label
Fixed the labels' data-list after adding the numbers shortcut keys for the tiers' selection (the labels options were not being changed)

6- Enable/Disable advanced options in the experiment and fixed specific view #68 That PR contains two main additions

A new configuration to the experiment to control whether you want to add the new features those added last year in that PR #60 so, at the adding new experiment we get a new field for the experiment's table called advanced annotation and at the creation of the experiment it appears as a checkbox.

At viewing a specific entry of the experiment files (the advanced features above were missing from that view, so they have been added and are being loaded and edited also from specific annotation view) (Example of before and after at an image category experiment)

After	Before

Coding period

Table of contents