Data Drift Function

This function helps to detect whether there is any drift in the data provided.

---

source

data_drift_test_selection

 data_drift_test_selection (user_input:int,
                            ref_data:pandas.core.frame.DataFrame,
                            cur_data:pandas.core.frame.DataFrame)

Selects the appropriate data drift test suite based on user input and runs the tests.

	Type	Details
user_input	int	User input to select the type of data drift test suite.
ref_data	DataFrame	Reference dataset.
cur_data	DataFrame	Current dataset.
Returns	TestSuite	The test suite containing the data drift tests.

---

source

detect_drift

 detect_drift (X:pandas.core.frame.DataFrame,
               window_size:Union[str,int]=None)

Detect data drift in a dataset.

	Type	Default	Details
X	DataFrame		Input dataset
window_size	typing.Union[str, int]	None	Window size for detecting data drift. Default is None.
Returns	typing.Generator[typing.Tuple[pandas.core.frame.DataFrame, pandas.core.frame.DataFrame], NoneType, NoneType]		Current chunk and next chunk of data.

---

source

run_data_drift_detection

 run_data_drift_detection ()

Runs the data drift detection on the dataset.

def main():
    """
    Main function to run the data drift detection.
    """
    run_data_drift_detection()

if __name__ == "__main__":
    main()

<generator object detect_drift>
Current Chunk:
                    Time  Electricity_load
0   2011-01-01 00:15:00              5.98
1   2011-01-01 00:30:00              8.45
2   2011-01-01 00:45:00              9.93
3   2011-01-01 01:00:00              6.92
4   2011-01-01 01:15:00              5.90
..                  ...               ...
95  2011-01-02 00:00:00             23.59
96  2011-01-02 00:15:00             30.22
97  2011-01-02 00:30:00             31.37
98  2011-01-02 00:45:00             30.89
99  2011-01-02 01:00:00             27.51

[100 rows x 2 columns]
Next Chunk:
                     Time  Electricity_load
100  2011-01-02 01:15:00             23.28
101  2011-01-02 01:30:00             30.18
102  2011-01-02 01:45:00             29.70
103  2011-01-02 02:00:00             24.97
104  2011-01-02 02:15:00             27.83
..                   ...               ...
195  2011-01-03 01:00:00             29.18
196  2011-01-03 01:15:00             30.69
197  2011-01-03 01:30:00             30.71
198  2011-01-03 01:45:00             28.21
199  2011-01-03 02:00:00             37.65

[100 rows x 2 columns]
------------------
Current Chunk:
                    Time  Electricity_load
0   2011-01-01 00:15:00              5.98
1   2011-01-01 00:30:00              8.45
2   2011-01-01 00:45:00              9.93
3   2011-01-01 01:00:00              6.92
4   2011-01-01 01:15:00              5.90
..                  ...               ...
95  2011-01-02 00:00:00             23.59
96  2011-01-02 00:15:00             30.22
97  2011-01-02 00:30:00             31.37
98  2011-01-02 00:45:00             30.89
99  2011-01-02 01:00:00             27.51

[100 rows x 2 columns]
Next Chunk:
                     Time  Electricity_load
200  2011-01-03 02:15:00             29.07
201  2011-01-03 02:30:00             36.21
202  2011-01-03 02:45:00             36.20
203  2011-01-03 03:00:00             37.40
204  2011-01-03 03:15:00             33.98
..                   ...               ...
295  2011-01-04 02:00:00             34.62
296  2011-01-04 02:15:00             39.64
297  2011-01-04 02:30:00             33.20
298  2011-01-04 02:45:00             33.99
299  2011-01-04 03:00:00             33.25

[100 rows x 2 columns]
------------------
Current Chunk:
                    Time  Electricity_load
0   2011-01-01 00:15:00              5.98
1   2011-01-01 00:30:00              8.45
2   2011-01-01 00:45:00              9.93
3   2011-01-01 01:00:00              6.92
4   2011-01-01 01:15:00              5.90
..                  ...               ...
95  2011-01-02 00:00:00             23.59
96  2011-01-02 00:15:00             30.22
97  2011-01-02 00:30:00             31.37
98  2011-01-02 00:45:00             30.89
99  2011-01-02 01:00:00             27.51

[100 rows x 2 columns]
Next Chunk:
                     Time  Electricity_load
300  2011-01-04 03:15:00             32.32
301  2011-01-04 03:30:00             39.05
302  2011-01-04 03:45:00             33.20
303  2011-01-04 04:00:00             40.69
304  2011-01-04 04:15:00             34.53
..                   ...               ...
395  2011-01-05 03:00:00             45.22
396  2011-01-05 03:15:00             44.52
397  2011-01-05 03:30:00             45.92
398  2011-01-05 03:45:00             43.94
399  2011-01-05 04:00:00             38.37

[100 rows x 2 columns]
------------------
Current Chunk:
                    Time  Electricity_load
0   2011-01-01 00:15:00              5.98
1   2011-01-01 00:30:00              8.45
2   2011-01-01 00:45:00              9.93
3   2011-01-01 01:00:00              6.92
4   2011-01-01 01:15:00              5.90
..                  ...               ...
95  2011-01-02 00:00:00             23.59
96  2011-01-02 00:15:00             30.22
97  2011-01-02 00:30:00             31.37
98  2011-01-02 00:45:00             30.89
99  2011-01-02 01:00:00             27.51

[100 rows x 2 columns]
Next Chunk:
                     Time  Electricity_load
400  2011-01-05 04:15:00             47.30
401  2011-01-05 04:30:00             43.42
402  2011-01-05 04:45:00             37.61
403  2011-01-05 05:00:00             47.38
404  2011-01-05 05:15:00             40.19
..                   ...               ...
495  2011-01-06 04:00:00             52.84
496  2011-01-06 04:15:00             54.99
497  2011-01-06 04:30:00             49.22
498  2011-01-06 04:45:00             50.45
499  2011-01-06 05:00:00             50.74

[100 rows x 2 columns]
------------------
Current Chunk:
                    Time  Electricity_load
0   2011-01-01 00:15:00              5.98
1   2011-01-01 00:30:00              8.45
2   2011-01-01 00:45:00              9.93
3   2011-01-01 01:00:00              6.92
4   2011-01-01 01:15:00              5.90
..                  ...               ...
95  2011-01-02 00:00:00             23.59
96  2011-01-02 00:15:00             30.22
97  2011-01-02 00:30:00             31.37
98  2011-01-02 00:45:00             30.89
99  2011-01-02 01:00:00             27.51

[100 rows x 2 columns]
Next Chunk:
                     Time  Electricity_load
500  2011-01-06 05:15:00             55.23
501  2011-01-06 05:30:00             51.88
502  2011-01-06 05:45:00             51.71
503  2011-01-06 06:00:00             46.29
504  2011-01-06 06:15:00             55.59
..                   ...               ...
595  2011-01-07 05:00:00             62.93
596  2011-01-07 05:15:00             63.00
597  2011-01-07 05:30:00             51.62
598  2011-01-07 05:45:00             55.13
599  2011-01-07 06:00:00             51.60

[100 rows x 2 columns]
------------------
Current Chunk:
                    Time  Electricity_load
0   2011-01-01 00:15:00              5.98
1   2011-01-01 00:30:00              8.45
2   2011-01-01 00:45:00              9.93
3   2011-01-01 01:00:00              6.92
4   2011-01-01 01:15:00              5.90
..                  ...               ...
95  2011-01-02 00:00:00             23.59
96  2011-01-02 00:15:00             30.22
97  2011-01-02 00:30:00             31.37
98  2011-01-02 00:45:00             30.89
99  2011-01-02 01:00:00             27.51

[100 rows x 2 columns]
Next Chunk:
                     Time  Electricity_load
600  2011-01-07 06:15:00             53.16
601  2011-01-07 06:30:00             53.75
602  2011-01-07 06:45:00             63.79
603  2011-01-07 07:00:00             50.58
604  2011-01-07 07:15:00             51.21
605  2011-01-07 07:30:00             65.69
606  2011-01-07 07:45:00             56.20
607  2011-01-07 08:00:00             56.15
608  2011-01-07 08:15:00             60.37
609  2011-01-07 08:30:00             54.09
610  2011-01-07 08:45:00             63.71
611  2011-01-07 09:00:00             57.42
612  2011-01-07 09:15:00             63.50
613  2011-01-07 09:30:00             63.27
614  2011-01-07 09:45:00             66.21
615  2011-01-07 10:00:00             62.27
616  2011-01-07 10:15:00             61.35
617  2011-01-07 10:30:00             63.00
618  2011-01-07 10:45:00             57.09
619  2011-01-07 11:00:00             62.45
620  2011-01-07 11:15:00             58.74
621  2011-01-07 11:30:00             57.34
622  2011-01-07 11:45:00             55.03
623  2011-01-07 12:00:00             55.42
624  2011-01-07 12:15:00             56.55
625  2011-01-07 12:30:00             59.36
626  2011-01-07 12:45:00             58.43
627  2011-01-07 13:00:00             57.35
628  2011-01-07 13:15:00             59.26
629  2011-01-07 13:30:00             69.89
630  2011-01-07 13:45:00             60.57
631  2011-01-07 14:00:00             69.65
632  2011-01-07 14:15:00             55.25
633  2011-01-07 14:30:00             55.65
634  2011-01-07 14:45:00             66.84
635  2011-01-07 15:00:00             54.47
636  2011-01-07 15:15:00             67.93
637  2011-01-07 15:30:00             58.61
638  2011-01-07 15:45:00             59.44
639  2011-01-07 16:00:00             61.80
640  2011-01-07 16:15:00             65.85
641  2011-01-07 16:30:00             60.27
------------------