Link Search Menu Expand Document

A Guide to Data Manipulation and Visualization

Convenience

This technical guide is aimed at researchers running experiments with Byteflies Kits. It discusses relevant Byteflies Cloud and API functionality, as well as basic data science instructions to visualize and review the recorded data. It is not a replacement for the main user manual which discusses how to safely operate Sensor Dots and their accessories.

Applicable Firmware version: 1.11.0
Last updated: June 26th, 2020

Table of contents
  1. Glossary
  2. Setting up a New Experiment in the Byteflies Cloud
    1. Before You Start
    2. Patient ID
    3. (Re)Configuring Sensor Dots
    4. Ready to Start?
  3. Managing Completed Recordings
    1. Accessing a Recording via the Web Interface
    2. Basic Data Plotting
    3. Accessing a Recording via the API

Glossary

Signal
A physiologic or motion signal, as recorded by a Sensor Dot (e.g. ECG).
Channel
A Sensor Dot can record two biopotential Signals simultaneously and has two pairs of electrode connectors that each form a Channel.
Recording
The combination of all data recorded by a single Sensor Dot is grouped in a Recording.
Group
A Group combines one or more Docking Stations that are assigned to a specific study.
Patient ID
A unique string used to identify a study subject.
Patient Record
A list of pre-configured Patient IDs and the hardware they are associated with.
User Account
A Byteflies Cloud account that can contain one or more Groups and can only be accessed with a specific username and password.

Setting up a New Experiment in the Byteflies Cloud

Byteflies Kits are set up in the factory for your study. However, for more advanced use cases, it may be necessary to change certain configuration settings. These can be accessed from your Byteflies Cloud account which was set up together with your Byteflies support contact.

Before You Start

Sensor Dots can be configured to measure two biopotential (ExG) Channels, specifically EEG, ECG and EMG. It is possible to select a different Signal for each Channel. The amplifiers in the Sensor Dot work differently depending on the selected Signal, so it is important to set them correctly.

If the configuration loaded on your Sensor Dots needs to be changed, follow the instructions below. It is currently not possible to check the active configuration of a Sensor Dot through the Byteflies Cloud web interface. For complex experiments with multiple Sensor Dots, it is a good idea to label them with a specific code. In case this information got lost, you can always reconfigure your Sensor Dots using he instructions below. This will not result in any loss of data.

Patient ID

You can use the Byteflies Cloud to configure a Patient ID to associate one or more Recordings with a specific study subject. Two methods are available:

A Recording was already made and is available in your Byteflies Cloud account

See Accessing a Recording via the Web Interface.

You configured a Patient Record before you started a new Recording

You can configure a Patient Record before starting a new Recording. Follow these steps:

  1. Log in to your Byteflies Cloud account and continue to your Group.
  2. Copy a Docking Station or Sensor Dot ID you wish to associate with a Patient ID a priori.
  3. Go back to the Group page and click the “Patient Management” icon to the right of your Group name.
  4. On the page that opens, set a Patient ID and paste the Docking Station or Sensor Dot ID(s) you just copied to associate it with that Patient ID. Set a start and end date for this association.
  5. For as long as the association is active, any Recordings uploaded via that Docking Station or made with that Sensor Dot will be automatically assigned to that Patient ID.

(Re)Configuring Sensor Dots

Go to the Group page on the Byteflies Cloud and click on the “Docking Stations” button on the right hand side. On the page that loads, click on the cog icon next to the Docking Station you plan to use, and follow the instructions on that page. Briefly:

  1. Select your desired configuration (ExG Channels and ACC and/or GYR) and hit “send”.
  2. Wait for a confirmation that the Docking Station received the new configuration.
  3. Dock the Sensor Dots you want to re-configure and hit “submit”. The LED will flash green briefly if the configuration was successful. If you receive an error state, try again. If it still is not working, please contact Byteflies support.

Note that currently configurations can only be changed one-by-one. For instance, if you want to configure 2 Sensor Dots as [Ch1: EEG, Ch2: EEG, ACC @50 Hz, GYR OFF] and 1 Sensor Dot as [Ch1: OFF, Ch2: OFF, ACC @100 Hz, GYR @100 Hz], send the first configuration and dock 2 Sensor Dots. When successful, first remove the Sensor Dots, set the second configuration and dock the final Sensor Dot.

Ready to Start?

Start each Sensor Dot you plan to use by lifting it from the Docking Station. Within 1 sec the Recording will start. To collect as much useful data as possible, it is good practice to make sure the cradles, patches and/or electrodes are already applied so you can pop the Sensor Dot immediately into the cradle. Detailed instructions are in the user manual and any specific information you may have received from your Byteflies support contact.

Each Sensor Dot will store an accurate timestamp when it starts a recording based on the internet connected clock of the Docking Station. In other words, it is not necessary to try to time the start of a Recording of multiple Sensor Dots with respect to each other.

Managing Completed Recordings

Once you have completed a Recording and placed the Sensor Dot(s) back in the Docking Station, the data will upload to your Byteflies Cloud account.

These files are quite large and this may take time depending on the speed of your internet connection. Please be patient for as long as the LED on the Sensor Dot(s) is/are orange.

Accessing a Recording via the Web Interface

From the Group page on the Byteflies Cloud, select the relevant Group (you most likely only have one), which will open a view of your recently completed Recordings.

You will find the following interface elements:

  • A new entry (line) for each uploaded Recording
  • A search box to filter your list of Recordings based on a specific query
  • A date range (by default, only Recordings dating back a week are shown)

Each Recording entry has information on:

  • The devices used to generate that recording
  • A Patient ID (if set, see Patient ID
  • Icons signifying which Signals were recorded*
  • The length of the Recording

Investigational Use Only

*Note that some of these icons will be color coded. This is an experimental feature that can be safely ignored unless your Byteflies support contact gave you specific instructions.


Clicking on one of the Recording entries will open its page. There you will find:

  • Detailed information on the hardware used
  • The Patient ID, which can be added or edited from this page
  • The Recording start and stop time in millisecond resolution; note that time timestamps will be shown in the timezone where the recording was made
  • Below, the Signals that are part of the Recording are listed:
    • For an ExG Signal, the Channel used is specified
    • The Conversion Factor converts the raw Signal into specific units (see table below)
    • The Sampling Rate specifies the sampling frequency; also refer to Advanced Options for more information
    • The file button allows you to download the associated Recording as a CSV file (see 2nd table below)

Investigational Use Only

Note that sometimes additional files may be generated (e.g. ARTFCTS, RPEAK); these are experimental and should not be used unless specifically instructed by your Byteflies support contact


Conversion Factors

Signal Multiplication Factor Final Units
EEG 0.4808 nanoVolts (nV)
ECG 1.4424 nanoVolts (nV)
EMG 0.4808 nanoVolts (nV)
ACC 0.0002 g-force (g)
GYR 0.0305 degrees/sec (deg/sec)

CSV format for ExG file

time channel
Time in seconds relative to the start of the recording (float) Raw signal value (integer)

CSV format for ACC and GYR files

time channel1 channel2 channel3
Time in seconds relative to the start of the recording (float) Raw signal value X-axis (integer) Raw signal value Y-axis (integer) Raw signal value Z-axis (integer)

Basic Data Plotting

There are a lot of possible ways to visualize the recorded data. For your information, we’ve included a number of examples using Python and Jupyter Notebooks below.

These examples use the time index in seconds which is stored as offset with regard to the start time for each sample. Note that it is possible to assume a fixed sampling frequency (e.g. 250 Hz for EEG) but when comparing Signals from more than one Sensor Dot, the time values will be more accurate.

The drift in a Sensor Dot for a 24 h Recording is < 3 seconds when using the time values.

Plot an EEG Recording (Single Sensor Dot)

This example assumes a dual Channel EEG recording was made with the accelerometer activated at 25 Hz.

import pandas as pd
import matplotlib.pyplot as plt
from datetime import datetime

# Load data (assuming the CSV file formating is exactly as it was downloaded from 
# Byteflies Cloud and the files are sitting in the same directory as your script or notebook)
eeg_ch1 = pd.read_csv("filename1.csv", names=["time", "raw"], skiprows=1)
eeg_ch2 = pd.read_csv("filename2.csv", names=["time", "raw"], skiprows=1)
eeg = pd.merge(eeg_ch1, eeg_ch2, on="time", suffixes=("_ch1", "_ch2"))
acc = pd.read_csv("filename3.csv", names=["time", "x", "y", "z"], skiprows=1)
eeg.head()

# Get the start timestamp for the Recording from the Recording detail page on the 
# Byteflies Cloud and fill it out below (YYYY,MM,DD,HH,MM,SS,uS) [uS = microseconds, 
# i.e. Cloud milliseconds * 1000] 
start_dt = datetime(2020,6,19,7,26,13,805000)
start_ts = start_dt.timestamp()

# Convert Signals to correct unit (note that this is a linear transformation and thus
# not strictly necessary)
eeg_list = eeg.keys()
acc_list = acc.keys()

for i in range(len(eeg_list)-1):
	eeg[eeg_list[i+1]] *= 0.4808  # To nV

for i in range(len(acc_list)-1):
	acc[acc_list[i+1]] *= 0.0002  # To g
	
# Convert the time values in seconds to a datetime index for easier plotting and Signal comparison
t_ns_eeg = (start_ts + eeg["time"])*1e9
eeg.index = t_ns_eeg.astype("datetime64[ns]")
eeg.drop(["time"], axis=1, inplace=True)

t_ns_acc = (start_ts + acc["time"])*1e9 
acc.index = t_ns_acc.astype("datetime64[ns]")
acc.drop(["time"], axis=1, inplace=True)

# Plot the data
fig, (ax1, ax2) = plt.subplots(2, 1, sharex=True, tight_layout=True)
ax1.plot(eeg, lw=1)
ax1.grid()
ax1.set_title("EEG")
ax1.set_ylabel(r"$\mu$V")
ax1.legend(["EEG Ch1", "EEG Ch2"], loc="best")

ax2.plot(acc, lw=1)
ax2.grid()
ax2.set_title("ACC")
ax2.set_ylabel("g")
ax2.legend(loc="best")
ax2.legend(["X-axis", "Y-axis", "Z-axis"], loc="best")

fig.autofmt_xdate()
plt.show()

Plot an ECG and EMG Recording (Dual Sensor Dot)

This examples assumes two Sensor Dots were used with only one ExG Channel in use each, and accelerometer activated at 25 Hz for both Sensor Dots.

import pandas as pd
import matplotlib.pyplot as plt
from datetime import datetime

# Load data (assuming the CSV file formating is exactly as it was downloaded from 
# Byteflies Cloud and the files are sitting in the same directory as your script or notebook)
ecg_ch1 = pd.read_csv("sensordot1_channel1.csv", names=["time", "raw"], skiprows=1)
emg_ch1 = pd.read_csv("sensordot2_channel1.csv", names=["time", "raw"], skiprows=1)
acc_sd1 = pd.read_csv("sensordot1_acc.csv", names=["time", "x", "y", "z"], skiprows=1)
acc_sd2 = pd.read_csv("sensordot2_acc.csv", names=["time", "x", "y", "z"], skiprows=1)
ecg_ch1.head()

# Get the start timestamp for the Recording from the Recording detail page on the 
# Byteflies Cloud and fill it out below (YYYY,MM,DD,HH,MM,SS,uS) [uS = microseconds, 
# i.e. Cloud milliseconds * 1000] 
start_dt = datetime(2020,6,19,7,26,13,805000)
start_ts = start_dt.timestamp()

# Convert Signals to correct unit (note that this is a linear transformation and thus 
# not strictly necessary)
ecg_ch1["raw"] *= 1.4424  # To nV
emg_ch1["raw"] *= 0.4808  # To nV
acc_list = acc_sd1.keys()

for i in range(len(acc_list)-1):
	acc_sd1[acc_list[i+1]] *= 0.0002  # To g
	acc_sd2[acc_list[i+1]] *= 0.0002  # To g
	
# Convert the time values in seconds to a datetime index for easier plotting and Signal comparison
sigList = [ecg_ch1, emg_ch1, acc_sd1, acc_sd2]

for i in range(len(sigList)):
	t_ns = (start_ts + sigList[i]["time"])*1e9
	sigList[i].index = t_ns.astype("datetime64[ns]")
	sigList[i].drop(["time"], axis=1, inplace=True)

# Plot the data
fig, (ax1, ax2, ax3) = plt.subplots(3, 1, sharex=True, tight_layout=True)
ax1.plot(ecg_ch1, lw=1)
ax1.plot(emg_ch1, lw=1)
ax1.grid()
ax1.set_title("ExG")
ax1.set_ylabel(r"$\mu$V")
ax1.legend(["ECG Ch1", "EMG Ch1"], loc="best")

ax2.plot(acc_sd1, lw=1)
ax2.grid()
ax2.set_title("ACC Sensor Dot 1")
ax2.set_ylabel("g")
ax2.legend(["X-axis", "Y-axis", "Z-axis"], loc="best")

ax3.plot(acc_sd2, lw=1)
ax3.grid()
ax3.set_title("ACC Sensor Dot 2")
ax3.set_ylabel("g")
ax3.legend(["X-axis", "Y-axis", "Z-axis"], loc="best")

fig.autofmt_xdate()
plt.show()

Advanced Options

If you cannot use the time column and absolutely require a completely fixed sampling frequency, you can calculate the closest approximation as follows:

# Assuming you already loaded a CSV file as demonstrated above
import numpy as np

fs = np.mean(1 / np.diff(ecg_ch1["time"]))

The uncertainty on the start timestamp for a specific Recording is less than 3 milliseconds. The drift of a single Sensor Dot is maximally 26 ppm (~2.25 seconds per 24 hours).

Accessing a Recording via the API

Downloading individual CSV files from the Byteflies Cloud interface works well for a relatively low number of recordings but it can get tedious rather quickly for larger studies. In those cases, it is easier to use the API directly from whatever programming environment you’re used to working.

The full API spec should have been provided to you as a separate file for reference but what’s below should already provide a practical example. All examples are in Python.

Log-in to your Byteflies Account

import requests

# Provide your Byteflies Cloud username and password in these variables
username = "<USERNAME>"
password = "<PASSWORD>" # NOTE: please retrieve the password from a safe place

# Request a temporary API token with your user credentials, to use in future requests
url = "https://cognito-idp.eu-west-1.amazonaws.com"
headers = {"X-Amz-Target":
       "AWSCognitoIdentityProviderService.InitiateAuth",
       "Content-Type": "application/x-amz-json-1.1"}
data = {"ClientId": "5kgqk7n83evq9cbqnam1biqnbr",
        "AuthFlow": "USER_PASSWORD_AUTH",
        "AuthParameters": {"USERNAME": username,
                           "PASSWORD": password
                            }
        }
response = requests.post(url, headers=headers, json=data)
idToken = response.json()["AuthenticationResult"]["IdToken"]

# Url of the Byteflies Cloud
url = "https://api.cloud.byteflies.net/"

# Request all the group this user has access to
headers = {"Authorization": idToken}
response = requests.get(url + "groups/", headers=headers)
groups = response.json()
print(groups)

If authentication is successful, you should get an answer from the API (list of JSON) that includes the name of all the Groups you have access to and their unique IDs:

[
    {
        "groupId": "1234",
        "groupName": "TestGroup",
        "groupDescription": "Description of group"
    }
    , ...
]

Get all the Recordings available in a Group

# Extract the Group Name of the group you'd like to request a recording from 
groupName = "<GROUPNAME>"  # NOTE: Should be filled in or requested from the user
for group in response.json():
    if group["groupName"] == groupName:
        groupID = group["groupId"]
        break
else:
    print("No group available with that name!")
    exit(1)

# List all recordings available in a Group
headers = {"Authorization": idToken}
response = requests.get(url + "groups/" + groupID + "/recordings", headers=headers)
recordings = response.json()
print(recordings)    

If the Group exists, you should get an answer from the API (list of JSON) that specifies a unique ID and other relevant information for your available Recordings:

[
    {
        "id": "33d8951c-999f-11e9-a2a3-2a2ae2dbcce4",
        "dockName": "DemoDock-8e6bde5ac7e7191caaf7e3c9",
        "dotId": "24:de:b6:f2:e3:60",
        "patient": "patient-ID",
        "startDate": 1526919822.1422584,
        "uploadDate": 1526939822.453,
        "signals": [...]
        "duration": 99.384483839
    }
    , ... 
]

Get the metadata for a specific Recording

# Retrieve the metadata of a specific Recording
recordingID = "<RECORDING ID>"  # NOTE: Should be filled in or requested from the user
headers = {"Authorization": idToken}
response = requests.get(url + "groups/" + groupID + "/recordings/" + recordingID, headers=headers)

if response.status_code != 200:  #if response is invalid, return an error
    print("Recording ID not found")

recordingMetadata = response.json()
print(recordingMetadata)

If the Recording exists, you should get an answer from the API (JSON) that contains the full metadata for that Recording and a list of available Signals in the Recording:

{
    "id": "33d8951c-999f-11e9-a2a3-2a2ae2dbcce4",
    "dockName": "DemoDock-8e6bde5ac7e7191caaf7e3c9",
    "dotId": "24:de:b6:f2:e3:60",
    "patient": "patient-ID",
    "startDate": 1526919822.1422584,
    "uploadDate": 1526939822.453,
    "signals": [
      {
            "type": "ACC",
            "rawData": "https://...",
            "samplingRate": "25",
            "conversionFactor":0.000244140625
        },
        {
            "type": "EEG",
            "channel": 1,
            "rawData": "https://...",
            "samplingRate": "250",
            "conversionFactor":0.4808108585052719,
            "algorithms":[
                {"type":"EEG_ARTFCTS",
                    "col":1,
                    "id":"09cdf800-7268-11ea-a8d6-a7682dc8f48c"}
                ]
        },
        {
            "type": "ECG",
            "channel": 2,
            "rawData": "https://...",
            "samplingRate": "250",
            "conversionFactor":1.4424325755158156
        },
        {
            "type": "GYR",
            "rawData": "https://...",
            "samplingRate": "200",
            "conversionFactor":0.030517578125
        }
    ],
    "duration": 99.384483839
}
, ... 

Get the CSV files for a Recording

# To download all CSV files for all Signals in a Recording, use the rawData property as download link
for signal in recordingMetadata["signals"]:
    response = requests.get(signal["rawData"], allow_redirects=True)
    open(recordingID+"_"+signal["type"]+"_"+str(signal["channel"])+".csv", 'wb').write(response.content)

If successful, this will download a CSV file for each Signal in the Recording to your current working directory as specified earlier.

Plot a Recording using Programmatic API

You can use the same code as specified above but you already have the timestamp and conversion factors available in your API payloads.

# Imports
...
# Load CSV files
...
# Get the start timestamp
start_ts = recordingMetadata["startDate"]

# Convert Signals to correct units
idx = 0
# Where [idx] is the index of the Signal as specified in `recordingMetadata`
cF_EMG = recordingMetadata["signals"][idx]["conversionFactor"]  
data["channel"] *= cF_EMG

# Plot the data
...

Copyright © 2020 Byteflies