Dataset processing

In this notebook we show several methods that can be used to process datasets (currently: qcodes.DataSet).

[1]:

import copy

import qtt
import qtt.utilities.visualization
from qtt.data import plot_dataset
from qtt.data import load_example_dataset

from qtt.dataset_processing import slice_dataset, dataset_dimension, average_dataset, process_dataarray

Load and plot a dataset

[2]:

dataset = load_example_dataset('elzerman_detuning_scan.json')

[3]:

qtt.data.plot_dataset(dataset)
_=qtt.utilities.visualization.plot_vertical_line(150e-6, color='b')
_=qtt.utilities.visualization.plot_vertical_line(350e-6, color='b')

../../_images/notebooks_datasets_example_dataset_processing_5_0.png

Slice a dataset

[4]:

loading_window = slice_dataset(dataset, [150e-6, 350e-6 ], axis=1)
plot_dataset(loading_window, fig=1)

detuning_window = slice_dataset(dataset, [0.05, 0.10 ], axis=0)
plot_dataset(detuning_window, fig=2)

../../_images/notebooks_datasets_example_dataset_processing_7_0.png

../../_images/notebooks_datasets_example_dataset_processing_7_1.png

Average over a dimension

[5]:

averaged_dataset = average_dataset(dataset, axis=0)
plot_dataset(averaged_dataset)

../../_images/notebooks_datasets_example_dataset_processing_9_0.png

Convert to dictionary format

[6]:

dataset_dictionary=qtt.data.dataset_to_dictionary(averaged_dataset)
print(dataset_dictionary)

{'extra': {'location': '2020-05-14/12-32-40_qtt_generic', '_version': '1.2.2'}, 'metadata': {}, 'arrays': {'signal': {'label': 'signal', 'name': 'signal', 'unit': 'a.u.', 'is_setpoint': False, 'full_name': 'signal', 'array_id': 'signal', 'shape': (1367,), 'ndarray': array([ 0.01288668, -0.29996808, -0.42453639, ...,  0.09534709,
        0.08637174,  0.08866183]), 'set_arrays': ('time',)}, 'time': {'label': 'time', 'name': 'time', 'unit': 's', 'is_setpoint': True, 'full_name': 'time', 'array_id': 'time', 'shape': (1367,), 'ndarray': array([0.00000000e+00, 1.02400054e-06, 2.04800108e-06, ...,
       1.39673671e-03, 1.39776070e-03, 1.39878469e-03]), 'set_arrays': ()}}}

Process a DataArray

[7]:

process_dataarray(dataset, 'signal', 'signal_squared', lambda x: x**2)
print(dataset)

plot_dataset(dataset, parameter_names=['signal', 'signal_squared'])

DataSet:
   location = '2020-05-14/12-32-33_qtt_generic'
   <Type>   | <array_id>     | <array.name>   | <array.shape>
   Setpoint | detuning       | detuning       | (240,)
   Measured | signal         | signal         | (240, 1367)
   Setpoint | time           | time           | (240, 1367)
   Measured | signal_squared | signal_squared | (240, 1367)

../../_images/notebooks_datasets_example_dataset_processing_13_1.png

Differentiate

This currently adds a new DataArray to the dataset.

[8]:

differentiated_dataset = qtt.data.diffDataset(dataset)
qtt.data.diffDataset(differentiated_dataset, 'x')
print(differentiated_dataset)

plot_dataset(differentiated_dataset, parameter_names=['diff_dir_x'])

DataSet:
   location = '2020-05-14/12-32-33_qtt_generic'
   <Type>   | <array_id>     | <array.name>   | <array.shape>
   Setpoint | detuning       | detuning       | (240,)
   Measured | signal         | signal         | (240, 1367)
   Setpoint | time           | time           | (240, 1367)
   Measured | signal_squared | signal_squared | (240, 1367)
   Measured | diff_dir_y     | diff_dir_y     | (240, 1367)
   Measured | diff_dir_x     | diff_dir_x     | (240, 1367)

../../_images/notebooks_datasets_example_dataset_processing_16_1.png

Serialization

We can convert a DataSet to a plain dictionary and back.

[9]:

dataset_dictionary = qtt.data.dataset_to_dictionary(dataset)
print(dataset_dictionary)
dataset2 = qtt.data.dictionary_to_dataset(dataset_dictionary)

{'extra': {'location': '2020-05-14/12-32-33_qtt_generic', '_version': '1.2.2'}, 'metadata': {}, 'arrays': {'detuning': {'label': 'detuning', 'name': 'detuning', 'unit': 'mV', 'is_setpoint': True, 'full_name': 'detuning', 'array_id': 'detuning', 'shape': (240,), 'ndarray': array([-0.2       , -0.19832636, -0.19665273, -0.19497909, -0.19330543,
       -0.19163179, -0.18995816, -0.18828452, -0.18661088, -0.18493724,
       -0.1832636 , -0.18158996, -0.17991632, -0.17824268, -0.17656904,
       -0.17489539, -0.17322175, -0.17154811, -0.16987447, -0.16820084,
       -0.1665272 , -0.16485356, -0.16317992, -0.16150628, -0.15983264,
       -0.158159  , -0.15648535, -0.15481171, -0.15313807, -0.15146443,
       -0.14979079, -0.14811715, -0.14644352, -0.14476988, -0.14309624,
       -0.1414226 , -0.13974896, -0.13807531, -0.13640167, -0.13472803,
       -0.13305439, -0.13138075, -0.12970711, -0.12803347, -0.12635984,
       -0.12468619, -0.12301255, -0.12133891, -0.11966527, -0.11799163,
       -0.11631799, -0.11464435, -0.11297071, -0.11129707, -0.10962343,
       -0.10794979, -0.10627615, -0.10460251, -0.10292887, -0.10125523,
       -0.09958159, -0.09790795, -0.09623431, -0.09456067, -0.09288703,
       -0.09121339, -0.08953975, -0.08786611, -0.08619247, -0.08451883,
       -0.08284519, -0.08117155, -0.07949791, -0.07782426, -0.07615063,
       -0.07447699, -0.07280335, -0.07112971, -0.06945607, -0.06778242,
       -0.06610879, -0.06443515, -0.06276151, -0.06108787, -0.05941423,
       -0.05774058, -0.05606695, -0.05439331, -0.05271966, -0.05104603,
       -0.04937239, -0.04769874, -0.0460251 , -0.04435147, -0.04267782,
       -0.04100418, -0.03933054, -0.0376569 , -0.03598326, -0.03430962,
       -0.03263598, -0.03096234, -0.0292887 , -0.02761506, -0.02594142,
       -0.02426778, -0.02259414, -0.0209205 , -0.01924686, -0.01757322,
       -0.01589958, -0.01422594, -0.0125523 , -0.01087866, -0.00920502,
       -0.00753138, -0.00585774, -0.0041841 , -0.00251046, -0.00083682,
        0.00083682,  0.00251046,  0.0041841 ,  0.00585774,  0.00753138,
        0.00920502,  0.01087866,  0.0125523 ,  0.01422594,  0.01589958,
        0.01757322,  0.01924686,  0.0209205 ,  0.02259414,  0.02426778,
        0.02594142,  0.02761506,  0.0292887 ,  0.03096234,  0.03263598,
        0.03430962,  0.03598326,  0.0376569 ,  0.03933054,  0.04100418,
        0.04267782,  0.04435147,  0.0460251 ,  0.04769874,  0.04937239,
        0.05104603,  0.05271966,  0.05439331,  0.05606695,  0.05774058,
        0.05941423,  0.06108787,  0.06276151,  0.06443515,  0.06610879,
        0.06778242,  0.06945607,  0.07112971,  0.07280335,  0.07447699,
        0.07615063,  0.07782426,  0.07949791,  0.08117155,  0.08284519,
        0.08451883,  0.08619247,  0.08786611,  0.08953975,  0.09121339,
        0.09288703,  0.09456067,  0.09623431,  0.09790795,  0.09958159,
        0.10125523,  0.10292887,  0.10460251,  0.10627615,  0.10794979,
        0.10962343,  0.11129707,  0.11297071,  0.11464435,  0.11631799,
        0.11799163,  0.11966527,  0.12133891,  0.12301255,  0.12468619,
        0.12635984,  0.12803347,  0.12970711,  0.13138075,  0.13305439,
        0.13472803,  0.13640167,  0.13807531,  0.13974896,  0.1414226 ,
        0.14309624,  0.14476988,  0.14644352,  0.14811715,  0.14979079,
        0.15146443,  0.15313807,  0.15481171,  0.15648535,  0.158159  ,
        0.15983264,  0.16150628,  0.16317992,  0.16485356,  0.1665272 ,
        0.16820084,  0.16987447,  0.17154811,  0.17322175,  0.17489539,
        0.17656904,  0.17824268,  0.17991632,  0.18158996,  0.1832636 ,
        0.18493724,  0.18661088,  0.18828452,  0.18995816,  0.19163179,
        0.19330543,  0.19497909,  0.19665273,  0.19832636,  0.2       ]), 'set_arrays': ()}, 'signal': {'label': 'signal', 'name': 'signal', 'unit': 'a.u.', 'is_setpoint': False, 'full_name': 'signal', 'array_id': 'signal', 'shape': (240, 1367), 'ndarray': array([[-0.04470825, -0.46432495, -0.56243896, ...,  0.08163452,
         0.04119873,  0.07385254],
       [-0.00152588, -0.33325195, -0.3894043 , ...,  0.15380859,
        -0.00259399,  0.08255005],
       [ 0.18203735, -0.2507019 , -0.31784058, ...,  0.00106812,
         0.04760742,  0.0227356 ],
       ...,
       [ 0.06408691, -0.4637146 , -0.40924072, ...,  0.2116394 ,
         0.16784668,  0.21575928],
       [ 0.01800537, -0.32394409, -0.49179077, ...,  0.12954712,
         0.03738403,  0.16204834],
       [ 0.21179199, -0.02929688, -0.46127319, ..., -0.06973267,
         0.10284424, -0.05752563]]), 'set_arrays': ('detuning', 'time')}, 'time': {'label': 'time', 'name': 'time', 'unit': 's', 'is_setpoint': True, 'full_name': 'time', 'array_id': 'time', 'shape': (240, 1367), 'ndarray': array([[0.00000000e+00, 1.02400054e-06, 2.04800108e-06, ...,
        1.39673671e-03, 1.39776070e-03, 1.39878469e-03],
       [0.00000000e+00, 1.02400054e-06, 2.04800108e-06, ...,
        1.39673671e-03, 1.39776070e-03, 1.39878469e-03],
       [0.00000000e+00, 1.02400054e-06, 2.04800108e-06, ...,
        1.39673671e-03, 1.39776070e-03, 1.39878469e-03],
       ...,
       [0.00000000e+00, 1.02400054e-06, 2.04800108e-06, ...,
        1.39673671e-03, 1.39776070e-03, 1.39878469e-03],
       [0.00000000e+00, 1.02400054e-06, 2.04800108e-06, ...,
        1.39673671e-03, 1.39776070e-03, 1.39878469e-03],
       [0.00000000e+00, 1.02400054e-06, 2.04800108e-06, ...,
        1.39673671e-03, 1.39776070e-03, 1.39878469e-03]]), 'set_arrays': ('detuning',)}, 'signal_squared': {'label': 'signal', 'name': 'signal_squared', 'unit': 'a.u.', 'is_setpoint': False, 'full_name': 'signal_squared', 'array_id': 'signal_squared', 'shape': (240, 1367), 'ndarray': array([[1.99882779e-03, 2.15597660e-01, 3.16337589e-01, ...,
        6.66419510e-03, 1.69733539e-03, 5.45419753e-03],
       [2.32830644e-06, 1.11056864e-01, 1.51635706e-01, ...,
        2.36570835e-02, 6.72880560e-06, 6.81451056e-03],
       [3.31375981e-02, 6.28514448e-02, 1.01022632e-01, ...,
        1.14087015e-06, 2.26646662e-03, 5.16907312e-04],
       ...,
       [4.10713255e-03, 2.15031230e-01, 1.67477969e-01, ...,
        4.47912375e-02, 2.81725079e-02, 4.65520658e-02],
       [3.24193388e-04, 1.04939775e-01, 2.41858163e-01, ...,
        1.67824561e-02, 1.39756594e-03, 2.62596644e-02],
       [4.48558480e-02, 8.58306885e-04, 2.12772959e-01, ...,
        4.86264471e-03, 1.05769373e-02, 3.30919866e-03]]), 'set_arrays': ('detuning', 'time')}, 'diff_dir_y': {'label': 'diff_dir_y', 'name': 'diff_dir_y', 'unit': None, 'is_setpoint': False, 'full_name': 'diff_dir_y', 'array_id': 'diff_dir_y', 'shape': (240, 1367), 'ndarray': array([[ 0.02467441,  0.02317244,  0.02378851, ..., -0.00312518,
        -0.00098616, -0.00793283],
       [ 0.0235369 ,  0.02001656,  0.01507552, ..., -0.00694358,
        -0.00068102, -0.007965  ],
       [ 0.01254013,  0.01410833,  0.0027335 , ..., -0.00931459,
        -0.00212355, -0.00249565],
       ...,
       [ 0.0170326 ,  0.01147505, -0.02129182, ..., -0.0197943 ,
         0.00859477, -0.00990452],
       [ 0.01980447,  0.03466332, -0.01652782, ..., -0.03326314,
         0.00596913, -0.02698303],
       [ 0.02312505,  0.04703598, -0.009843  , ..., -0.03772412,
         0.0043794 , -0.03545641]]), 'set_arrays': ('detuning', 'time')}, 'diff_dir_x': {'label': 'diff_dir_x', 'name': 'diff_dir_x', 'unit': None, 'is_setpoint': False, 'full_name': 'diff_dir_x', 'array_id': 'diff_dir_x', 'shape': (240, 1367), 'ndarray': array([[-6.75441402e-03, -1.24756511e-02, -1.79746588e-02, ...,
        -2.15791755e-03, -2.39050588e-03, -2.04945350e-03],
       [-6.93263137e-03, -1.22906849e-02, -1.75441497e-02, ...,
        -2.50083812e-03, -2.57659379e-03, -2.06919546e-03],
       [-4.35955638e-03, -7.98710219e-03, -1.18201167e-02, ...,
        -1.93977093e-03, -1.46484033e-03, -7.57196835e-04],
       ...,
       [-7.32889263e-03, -9.56584479e-03, -9.78552642e-03, ...,
         1.50182692e-03,  8.72919536e-04, -7.45698221e-05],
       [-5.63340545e-03, -8.27244296e-03, -9.04882311e-03, ...,
        -7.22117700e-05, -1.01094416e-03, -2.03497390e-03],
       [-4.53192539e-03, -7.33912314e-03, -8.56147986e-03, ...,
        -8.64665655e-04, -2.00935238e-03, -3.08634345e-03]]), 'set_arrays': ('detuning', 'time')}}}

[ ]: