|
|
@@ -1,6 +1,10 @@
|
|
1
|
1
|
# ampli
|
|
2
|
2
|
|
|
3
|
|
-_Repository for Petra's work at [ampli](https://www.ampli.nz) Jan-Feb 2019_
|
|
|
3
|
+_Repository for Petra Lamborn's internship at [ampli](https://www.ampli.nz) Jan-Feb 2019_
|
|
|
4
|
+
|
|
|
5
|
+This repository serves as both a reference for the folk at ampli to build upon what I produced for them during this internship, and as a CV item for me. Non-ampli people will not be able to directly use all but a handful of the below, but it still demonstrates what I can do.
|
|
|
6
|
+
|
|
|
7
|
+The purpose of this project was to cluster the electricity demand time series of a large number of ICPs (Installation Control Points), and then project the patterns for these clusters into the future.
|
|
4
|
8
|
|
|
5
|
9
|
## What isn't in this repository
|
|
6
|
10
|
|
|
|
@@ -18,7 +22,9 @@ This file is based on an example from [the postgresSQL website](http://www.postg
|
|
18
|
22
|
|
|
19
|
23
|
## SQL
|
|
20
|
24
|
|
|
21
|
|
-I've included a bunch of annotated SQL queries in `sql/queries.pgsql` and `sql/weather.pgsql`. The latter file is specifically about the weather dataset; the former is more general. Some of the queries are copied into the `py/util.py` file, but changing these two files will do nothing on their own.
|
|
|
25
|
+I've included a bunch of annotated SQL queries in `sql/queries.pgsql` and `sql/weather.pgsql`. The latter file is specifically about the weather dataset; the former is more general. Some of the queries are copied into the `py/util.py` file and similar, but changing these two files will do nothing on their own.
|
|
|
26
|
+
|
|
|
27
|
+The `queries.pgsql` file includes the commands used to create the tables available in the `--table` option of the `downkwh.py` script.
|
|
22
|
28
|
|
|
23
|
29
|
## Python
|
|
24
|
30
|
|
|
|
@@ -34,6 +40,7 @@ I've included a bunch of annotated SQL queries in `sql/queries.pgsql` and `sql/w
|
|
34
|
40
|
* scipy
|
|
35
|
41
|
* seaborn
|
|
36
|
42
|
* statsmodels
|
|
|
43
|
+* tabview
|
|
37
|
44
|
|
|
38
|
45
|
Virtual environments are loaded with `source venv/bin/activate`. The python scripts are in the `py/` folder. The scripts that are designed to be called directly are called by `python <scriptname.py>`; use `python <scriptname.py> -h` to view help. Note that most options will have a default, which may not be what you want, so always check.
|
|
39
|
46
|
|
|
|
@@ -239,12 +246,16 @@ Black is observed values; blue is harmonic trend; green is residuals/de-trended
|
|
239
|
246
|
|
|
240
|
247
|
The justification for this is that we want to include the deviation from typical value for that time of year in our model, not the value itself.
|
|
241
|
248
|
|
|
|
249
|
+As written, this script assumes a python virtual environment located in `../venv/`.
|
|
|
250
|
+
|
|
242
|
251
|
### `combmodels.R`
|
|
243
|
252
|
|
|
244
|
253
|
Taking `../data/9-clusters.agg.pkl` and `../data/weatherharm.csv` as inputs, this script first fits a harmonic model to cluster 1 and tests it with the assigned clusters in `../data/1617-agg.pkl`. It then fits models to all clusters and saves them collectively in `../models/1kmods.rds`.
|
|
245
|
254
|
|
|
246
|
255
|
Similar to the previous, some plots are again commented out.
|
|
247
|
256
|
|
|
|
257
|
+As written, this script assumes a python virtual environment located in `../venv/`.
|
|
|
258
|
+
|
|
248
|
259
|
## Shiny App
|
|
249
|
260
|
|
|
250
|
261
|
The [R Shiny](https://shiny.rstudio.com/) app is located in `shiny/app.R`. It requires the outputs from `weathmod.R` and `combmodels.R`, along with a couple of extra packages, installed with:
|
