Browse Source

Add and reorganise notes

Petra Lamborn 2 years ago
parent
commit
08c01a21b0
1 changed files with 26 additions and 79 deletions
  1. 26
    79
      notes.md

+ 26
- 79
notes.md View File

@@ -57,7 +57,9 @@ SELECT * FROM public.coup_tall_april limit 10;
57 57
 
58 58
 This dataset includes 34278 distinct ICPs: `SELECT COUNT(DISTINCT icp_id) FROM public.coup_tall_april;`
59 59
 
60
-Not every day has the same number of ICPs recorded for it(?):
60
+Not every day has the same number of ICPs recorded for it:
61
+
62
+(Some are added and removed each day; some are disconnected for various reasons on different days
61 63
 
62 64
 ```sql
63 65
 
@@ -67,42 +69,11 @@ GROUP BY read_date;
67 69
 
68 70
 ```
69 71
 
70
- read_date  | d_icp
71
-------------+-------
72
- 2017-04-01 | 34080
73
- 2017-04-02 | 34070
74
- 2017-04-03 | 34082
75
- 2017-04-04 | 34085
76
- 2017-04-05 | 34083
77
- 2017-04-06 | 34078
78
- 2017-04-07 | 34084
79
- 2017-04-08 | 34085
80
- 2017-04-09 | 34079
81
- 2017-04-10 | 34097
82
- 2017-04-11 | 34102
83
- 2017-04-12 | 34095
84
- 2017-04-13 | 34127
85
- 2017-04-14 | 34127
86
- 2017-04-15 | 34128
87
- 2017-04-16 | 34122
88
- 2017-04-17 | 34119
89
- 2017-04-18 | 34161
90
- 2017-04-19 | 34178
91
- 2017-04-20 | 34181
92
- 2017-04-21 | 34190
93
- 2017-04-22 | 34187
94
- 2017-04-23 | 34178
95
- 2017-04-24 | 34190
96
- 2017-04-25 | 34180
97
- 2017-04-26 | 34199
98
- 2017-04-27 | 34193
99
- 2017-04-28 | 34194
100
- 2017-04-29 | 34179
101
- 2017-04-30 | 34162
102
-
103 72
 
104 73
 Days have similar averages (within the same month), but sometimes values are negative:
105 74
 
75
+Negative values possibly from solar power injecting back into the network, but is so rare it can be ignored.
76
+
106 77
 ```sql
107 78
 
108 79
 SELECT read_date, min(kwh_tot), Avg(kwh_tot), max(kwh_tot) 
@@ -111,39 +82,6 @@ GROUP BY read_date;
111 82
 
112 83
 ```
113 84
 
114
- read_date  |   min   |          avg           |        max
115
-------------+---------+------------------------+--------------------
116
- 2017-04-01 |     0.0 | 0.41225447048611111114 |             30.928
117
- 2017-04-02 |     0.0 | 0.42826891265531748365 |             28.153
118
- 2017-04-03 |     0.0 | 0.43139900216145374882 |             28.041
119
- 2017-04-04 |     0.0 | 0.44293095264290254757 |             31.111
120
- 2017-04-05 |     0.0 | 0.44780382081976351847 | 29.100999999999999
121
- 2017-04-06 |     0.0 | 0.43275886569047479314 |             28.067
122
- 2017-04-07 |     0.0 | 0.42198233958748973126 |             37.413
123
- 2017-04-08 |     0.0 | 0.42289754718595667695 |             25.908
124
- 2017-04-09 |     0.0 | 0.43495351487230650355 |             30.373
125
- 2017-04-10 |     0.0 | 0.42881511692133227754 |             26.791
126
- 2017-04-11 |     0.0 | 0.42526183459425644637 |             35.234
127
- 2017-04-12 | -30.530 | 0.44193485420638412283 |             29.818
128
- 2017-04-13 |     0.0 | 0.44908520990222795247 |             31.721
129
- 2017-04-14 |     0.0 | 0.43110074745314071950 |             27.167
130
- 2017-04-15 |     0.0 | 0.41132879527074542899 |             30.746
131
- 2017-04-16 |     0.0 | 0.41155711552175527034 |             26.713
132
- 2017-04-17 |     0.0 | 0.42657600420586769836 |             27.751
133
- 2017-04-18 |     0.0 | 0.43113055579949845344 |             34.414
134
- 2017-04-19 |     0.0 | 0.43415263473579495584 |             26.547
135
- 2017-04-20 |     0.0 | 0.43513854431799342716 |             27.124
136
- 2017-04-21 |     0.0 | 0.43824147350589841087 |             30.365
137
- 2017-04-22 |     0.0 | 0.42576349014245180919 |             31.112
138
- 2017-04-23 |     0.0 | 0.44098844956307176160 |             31.099
139
- 2017-04-24 |     0.0 | 0.43441511284976113876 |             27.109
140
- 2017-04-25 |     0.0 | 0.43805378632728691245 |             25.776
141
- 2017-04-26 |     0.0 | 0.44017528594890688813 |             26.907
142
- 2017-04-27 |     0.0 | 0.44285652216828005733 |             29.544
143
- 2017-04-28 |     0.0 | 0.43437889688249400482 |             29.598
144
- 2017-04-29 | -31.624 | 0.45286084474384856201 |             31.874
145
- 2017-04-30 |     0.0 | 0.46408289668832816191 |             31.960
146
-
147 85
 Three values in this table are negative:
148 86
 
149 87
 ```sql
@@ -168,15 +106,24 @@ SELECT DISTINCT icp_id FROM public.coup_tall_april WHERE icp_id LIKE '%17' ORDER
168 106
 
169 107
 ```
170 108
 
171
- icp_id
172
----------
173
- I000117
174
- I000217
175
- I000417
176
- I000517
177
- I000617
178
- I000817
179
- I001117
180
- I001217
181
- I001317
182
- I001417
109
+This could be useful for sampling later.
110
+
111
+# Research
112
+
113
+## Roelofsen, Time Series Clustering
114
+
115
+A master's thesis from the Netherlands, which gives a lot of useful information. ([link](https://beta.vu.nl/nl/Images/stageverslag-roelofsen_tcm235-882304.pdf).)
116
+
117
+Notes in notebook; copy next week. Up to page 54.
118
+
119
+## Alonso, Time Series Clustering
120
+
121
+A presentation, which I still need to read. ([link](http://halweb.uc3m.es/esp/Personal/personas/amalonso/esp/ASDM-C02-clustering.pdf))
122
+
123
+## Twoards Data Science, Playing with time series data in python
124
+
125
+([link](https://towardsdatascience.com/playing-with-time-series-data-in-python-959e2485bff8).)
126
+
127
+## Minnaar, Time Series Classification and Clustering with Python
128
+
129
+http://alexminnaar.com/time-series-classification-and-clustering-with-python.html