This is the general discussion pad for the Hacking Ecology Mini-Hackathon at the 36c3.
Please list what you are working on, so that others do not so the same.
If you are implementing new datasets into videt, please look for APIs or FTP servers that host the data to avoid the necessity to ship datasets with our project.
Thank you all for joining this pad! And please, if you think that any of the decisions made until now was bad, please let me know or improve it! (Theo)
HEY! Cristian here (we just talked). Since this wants to be a self-contained project, we should add something to keep track of the dependencies it has. The simplest way of doing this is by dumping them into a "requirements.txt" file.
There are some new exotic tools like pipenv or poetry, but we could start with that. A "setup.py" file would also be benefic as an install / configuration script of the tool.
-> Is anybody already working on the requirements.txt?
-> Yes
Suggestion: let's use some existing data packaging standard, f.e:
- https://github.com/frictionlessdata/datapackage-py (+ https://github.com/rgieseke/pandas-datapackage-reader)
- https://github.com/intake/intake
Problem statement: There's a lot of data, it's all heterogeneous. How do we get it together into one place and allow people to contribute their own data?
============
Architecture 101
============
========
Initial steps
========
Please list what you are working on, so that others do not so the same.
If you are implementing new datasets into videt, please look for APIs or FTP servers that host the data to avoid the necessity to ship datasets with our project.
Thank you all for joining this pad! And please, if you think that any of the decisions made until now was bad, please let me know or improve it! (Theo)
HEY! Cristian here (we just talked). Since this wants to be a self-contained project, we should add something to keep track of the dependencies it has. The simplest way of doing this is by dumping them into a "requirements.txt" file.
There are some new exotic tools like pipenv or poetry, but we could start with that. A "setup.py" file would also be benefic as an install / configuration script of the tool.
-> Is anybody already working on the requirements.txt?
-> Yes
Suggestion: let's use some existing data packaging standard, f.e:
- https://github.com/frictionlessdata/datapackage-py (+ https://github.com/rgieseke/pandas-datapackage-reader)
- https://github.com/intake/intake
Problem statement: There's a lot of data, it's all heterogeneous. How do we get it together into one place and allow people to contribute their own data?
============
Architecture 101
============
- What kind of solution do we go for?
- A: Web-based
- What kind of backend and what stuff do we provide for visualization?
- MVP: what will it define?
- pull data from something like a CSV
- transform it and store it locally in a common format
- allow users to view that specific
========
Initial steps
========
- Define atomic work tasks
- Establish a way of working on the project and
- Setup a roadmap and a timeline
- Check the market!!! There may already be something similar available out of the box.