Filterpanel
Public Data
Private Data
Examples
Olympic Medals
- CSV file from data.world
- uses data released by The Guardian
- hosted on Kaggle and many other sites
- covers Olympic medals up to 2014
Shapes
- Trivial example using a JSON literal as input.
- Click a few checkboxes and watch how the "in" and "out" masks change.
Sample records
Movie
title | The Shawshank Redemption
year | 1994
decade | 1990
rated | R
released | 14 Oct 1994
runtime | 142 min
genre | Crime, Drama
director | Frank Darabont
writer | Stephen King (short story ""Rita Hayworth and Shawshank Redemption""),
Frank Darabont (screenplay)
actors | Tim Robbins, Morgan Freeman, Bob Gunton, William Sadler
plot | Two imprisoned men bond over a number of years, finding solace and eventual
redemption through acts of common decency.
language | English
country | USA
awards | Nominated for 7 Oscars. Another 19 wins & 30 nominations.
poster | https://images-na.ssl-images-amazon.com/images/I/71TO64qm%2BbL._AC_SY741_.jpg
ratings_source | Internet Movie Database
ratings_value | 9.3/10
metascore | 80
imdbrating | 9.3
imdbvotes | 1,825,626
imdbid | tt0111161
dvd | 27 Jan 1998
boxoffice | N/A
production | Columbia Pictures
website | N/A
response | True
tomatourl | http://www.rottentomatoes.com/m/shawshank_redemption/Olympic Medal
+--------+------+--------+----------+------------+-----------------+--------------+--------+----------------+-------+---------------+ | Season | Year | City | Sport | Discipline | Athlete | Country_code | Gender | Event | Medal | Country | +--------+------+--------+----------+------------+-----------------+--------------+--------+----------------+-------+---------------+ | summer | 2012 | London | Aquatics | Swimming | PHELPS, Michael | USA | Men | 100M Butterfly | Gold | United States | +--------+------+--------+----------+------------+-----------------+--------------+--------+----------------+-------+---------------+
Public vs Private
When you open a file on your computer, the data will be read into Filterpanel running on your browser but will not be transmitted over the internet.
There is no Filterpanel server. There are no accounts. There is no database. Nothing is stored. It's just a script. The entire CSV file is read on startup every time, either from local storage or the internet. The second time is faster, because caching.
Github
filterpanel
https://github.com/johndimm/filterpanelfilterpanel-example
https://github.com/johndimm/filterpanel-examplefilterpanel-csv
https://github.com/johndimm/filterpanel-csvWhat is a filter panel?
Filterpanels are used in ecommerce to deal with the problem of vague queries that produce too much data. The user needs a way to wade through the thousands of items that match the query "tv". The filter panel appears on the left side of the page after a query. Check the boxes to reduce the list.
How do you use it?
The panels is a series of feature filters. You check the values you want to include in the list of items on the right. If a filter has nothing checked, it's as if everything is checked and all values are allowed. You check multiple boxes in various filters until the list shows items with the features you care about.How does it work?
The standard implementation reacts to a click by generating a new page of data in the database running on the server, then download and render it on the client. This implemention downloads the data to the client at the outset, so it responds to clicks locally and immediately.
The algorithm uses document masks to represent the actions of filters. The filters have no knowledge of each other, and the container app blindly aggregates and disseminates the results of the filters. The technique makes high demands on the network, the local machine, and the browser, compared to the standard solution. But compared to games or video, not so much.
What kinds of data work well?
See the examples, Movies in particular.- Each row has value on its own. Rows do not need aggregation to become interesting.
- Many columns are categorical: they have a small number of repeating values. It's even better if the values are text.
- The categorical fields are shown in increasing cardinality. So rated and genre on Movies show up before country and language.
- The technique works well on rows up to the 5,000 to 10,000 range. For larger datasets, it is best to feed the filterpanel with the results of a query.
Analysts, Product Designers, Casual Readers
The best case for this UI may be product catalogs, but lists of anything with known categorical attributes can benefit.
First, try it out on a local CSV file. If the results are promising, you may want to edit your spreadsheet to remove boring columns or make other changes to improve the filterpanel display.
Hosting your data on data.world
If it looks good and you want to share a local CSV file as rendered by Filterpanel, you need to make it available on the internet or intranet using a url. One option for public data is to upload it to data.world. See the Movies and Olympics examples above for details.
Hosting your data on Google Sheets
Here is a google sheet that is publicly available:
https://docs.google.com/spreadsheets/d/1obx8JHesu-FGVUKsm6cX-Use9V8PZx-yqJmJHu095DE/edit#gid=0To make it available as CSV, you need to edit the url like this:
https://docs.google.com/spreadsheets/d/1obx8JHesu-FGVUKsm6cX-Use9V8PZx-yqJmJHu095DE/export?format=csvThat's the url you load into filterpanel, resulting in this final url. It shows the Google Sheets data as it is rendered by the filterpanel UI.
https://filterpanel-csv.vercel.app/FetchCSV?url=https%3A%2F%2Fdocs.google.com%2Fspreadsheets%2Fd%2F1obx8JHesu-FGVUKsm6cX-Use9V8PZx-yqJmJHu095DE%2Fexport%3Fformat%3Dcsv
Using data from OpenML
OpenML has data:https://www.openml.org/s/88/dataLet's try cars. Right-click on the CSV link at the top right and select Copy link.
https://www.openml.org/data/get_csv/4965299/cars1.arffPaste that into filterpanel.
https://filterpanel-csv.vercel.app/FetchCSV?url=https%3A%2F%2Fwww.openml.org%2Fdata%2Fget_csv%2F4965299%2Fcars1.arff+Unfortunately, filterpanel hangs, and the javascript console says:
Access to fetch at 'https://www.openml.org/data/get_csv/4965299/cars1.arff' from origin 'https://filterpanel-csv.vercel.app' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.The solution is a small app that bypasses cors: https://powerful-reef-24520.herokuapp.com/
Using the proxy, the final url for filterpanel is this:
http://localhost:3001/FetchCSV?url=https%3A%2F%2Fpowerful-reef-24520.herokuapp.com%2Fhttps%3A%2F%2Fwww.openml.org%2Fdata%2Fget_csv%2F4965299%2Fcars1.arff
Bring Your Own Data
The goal was to do two things:
- provide a data visualization tool you can use on your own local data
- allow you to post a url that shows your data in the same visualization tool
These requirement seem to call for user accounts and online data storage, with the associated security issues.
We avoid it all by using either a local file as source, in which case the file is read locally and processed locally in javascript, or an online source in the form of a url that downloads a CSV file.
That's why you don't have to authenticate yourself to use filterpanel. There is no website. This is purely a free script that runs on your computer. The catch is that you have to Bring Your Own Data.Developers
The npm package uses React, so if you have some boring grids and are using React, you are in luck my friend! See the filterpanel-example app for a simple integration using JSON arrays, in React/Nextjs.
Copyright (c) John Dimm 2021