City Fish

Dummy blogpost

2022-09-26T18:04:00-04:00

Ping ping! this is an initial dummy blogpost. Let's see how this will work on deployment

Dataframe Schema

2022-09-25T00:20:00-04:00

DataFrame Schema is a small and slim package that does only one job - it checks if a given DataFrame fits certain list of defined expectation. It is heavily inspired by jsonschema, has only one dependency (Pydantic) and tries to be as simple and small as possible.

In our experience, we found that defining a very simple, explicit, and easy-to-generate definition of what we expect to get, is crucial in our day-to-day work.

Core Ideas

Compared to the alternatives, this package has few cornerstone ideas: 1. It is meant to be simple and easy to use, taking minimal time to use and set up. 2. It puts Dataframe front and center and goes from there (Read more below) 3. It will be interchangeable and will try to read from and write to other formats (e.g. tableschema, great_expectations, etc).

DataFrame in the middle

We explicitly define "DataFrame" as a package-agnostic abstraction, as it (theoretically) could be a dataframe from pandas,, or geopandas, or dask, or ray, or Spark, or anything else. In each case, we use corresponding flavor, decision and assumptions made by the corresponding package. For example, if we check data derive from the database into pandas, we only care about datatype defined by pandas, and let it infer datatypes as it wish.

Right now we only support pandas, but it should be easy to add other flavors and it is on our roadmap.

Resources

MosPlus (PlutoPlus copycat)

2020-09-27T20:58:00-04:00

MosPlus is a forked version of Chris Wong's awesome PlutoPlus project, built using open footprints for Moscow (Russia).

Building Footprint is a great Moscow Open Data Resource that contains a wealth of information about the city's building footprints, including address, cadaster zoning, status, registration data, and few more attributes.It contains information for the city's 145,000+ buildings, and includes 19 attributes for each one. That is (so far) a unique open data collection for Russia!

Moscow Building Footprint is quite large, available only through API and hard to use. That is why I forked and edited blueprints for PlutoPlus, a great tool from Chris Wong (originally for MapPluto dataset) to to help people get access to smaller chunks of the data quickly and easily for whatever they are working on. All data is version from 25.03.2016 and can be exported as geoJSON, zipped shapefile, and CSV, or can be imported directly to your cartoDB account. Geometries are exported in WGS84 (Latitude and Longitude). For neighborhood (rayon) borders, I used this dataset from Gis-Lab.

Note: FYI, There is a separate, not-exactly-opensourced yet invaluable dataset in the wild, data for ALL Buildings in Russia including address, year, type, number of units and floors - so pretty similar to Pluto, originated on reforma-zkh website. It was since removed, but can be found on the internet.

Links

Data

Projects I worked on at Ria Novosti (2013-2015)

2020-05-02T00:20:00-04:00

I've been working at Ria Novosti's Infographics team as a data journalist/editor from 2012 to 2015. That was a pivotal moment for me as I "officially" left architectural career towards more data-related. It was a great experience for many reasons, including quick turn-around project management, collaboration with developers and designers, quite creative data analytics, time and opportunity to read a lot, learn a lot, and meet with many interesting people.

Despite being a (one of two) official Russian news agencies, our team was quite liberal both as a collective, and also in terms of work we've done. This wasn't exactly a coincidence - on one side, it is really hard to read and write news, and not to make your own opinions. But it also was a deliberate policy of the agency, as we played along with the "liberal" wing of the government (and yes, also expand our audience). Our work was, in part, a reason for the whole agency to be dismantled in 2015; Our team was "sold" to Rambler Media Group. Despite all, I think that we've done a lot, and I am very proud of our job.

Where did they go? How Russian Deputies changed their political alignment

Project dedicated to 20th anniversary of Russian Duma (Parliament), tracing back the political alignment of deputies and all Political Parties ever present in there.

Team:

code: Michail Dunayev design: Valeriy Borisov editor, analyst: Philipp Kats director: Maya Stravinskaya

Clustering Russian Deputies

Clustering Russian Deputies is a project done in 2013 at Ria Novosti Studio of Infographics. Project explores how actual votes of Russian deputies are revealing underlying political structures and fractions, and how that behavior changes based on the topics of the laws. It also highlights how specific deputies voted on key issues, such as Prohibition of international adoption of Russian children, and some others.

Unfortunately, project itself is removed from the site now, but here is a review (and archived screenshot) - description

Team:

code: Evgeny Panov design: Alexey Novichkov, Valeriy Borisov editor, analyst: Philipp Kats director: Maya Stravinskaya

Deputies Tax Declarations "Calculator"

A visual representation of Deputies' tax declaration and distribution - who owns what and how much of it. Data was collected, parsed and cleaned. Car prices were estimated using external databases

Russian Government Budget "Calculator"

Visual representation of the state budget, and a "minigame" / survey on your personal preferences. For each custom budget, a country with similar budget strategy is shown.

Pension Reform Calculator (How they got you)

A visual representation of a new "Formula" for official pension (social security payments). Shows how it results in the same amount "on paper", allowing government to pay less over time.

Singapore Urban Forum

(Information is beautiful Award) link)

4 Posters, telling the story or Moscow's urban development

Design: Nadezhda Andrianova, Maria Miahaylova Data, Content: Philipp Kats

Other Projects

2020-01-01T00:00:00-05:00

This is an index of a few of my other projects. At some point I'll move each of them into a separate page and provide more details.

PyCombo - Python Wrapper around Combo Network partition Algorithm
pdvega, vega driver for pandas
StreetEasy Data Dashboard (Data Visualisation by Paul Buffa)