NYC Workshop Notes, May 2014
- 1 Measurement Interests and Prospective Methodologies
- 2 Measurement Platforms and Initiatives
- 3 Infrastructure and Testing Consideration
- 4 Initial Plans and Subjects for Discussion
- 5 Milestones
- 5.1 OONI
- 5.2 ICLab
- 5.3 CitizenLab
- 5.4 Least Authority
- 5.5 Encore
- 5.6 My Speed Test
- 5.7 Satellite
- 5.8 Measurement Lab
- 6 Other Documentation
Measurement Interests and Prospective Methodologies
- Search Manipulation
- Self-Censorship and Intimidation
- Technical Ineptitude and Compatibility Failures
- Physical Attacks
- End-Point Manipulation
Measurement Platforms and Initiatives
- Open Observatory of Network Interference (OONI)
- Measurement Lab
- Network Diagnostic Test
Infrastructure and Testing Consideration
Initial Plans and Subjects for Discussion
Thursday Morning Session Notes
- No one should be able to infer political intent because of the performance of censorship measurement tests:
- Should be similar to motivations regarding network neutrality; or,
- Why don’t conservative Christians organizations measure censorship?
- Clear need across projects and research applications for longitudinal standards and baseline, to allow for framing in terms of Internet health or differential access changes;
- Should take the politics out of measurement, which occurs especially when measurements only target the Middle East and around particular events;
- Units of reporting or observation matter since censorship occurs at lower levels than a country-basis, schools and organization censor at the lowest level;
- Remaining questions across projects about the level of reporting: raw data versus metadata versus abstracted information;
- Destigmatization may be partially accomplishable through labelling and language, words such as health and interference (latter seen with the naming of OONI):
- Dry language is helpful toward this purpose, but not as motivating or funded as ‘sexier’ branding of censorship,
- Speaking in terms of a gradient versus a binary or dichotomous situation as well;
Friday Morning Session
- On the use of rPis, presentation matters, BISmark trials had resistance based on the use of a grey box -- problematic since people expect a network device to look a certain way,
- This may not be an issue since the incentive is different (free of expression appeal versus a free device);
- Requests for dashboard to demonstrate the impact of participate in measurement efforts, could be linkable with measurement IDs without deanonymization,
- Projects may need fresh documentation review by others not versed in the project’s development,
- Map expectations ahead of time for testing and development of projects,
- Need for modular infrastructure for helpers, collectors -- however, changes in infrastructure have already burned projects,
- Including helper stability and compatibility;
- Canonical DNS answers and mapping will help with assessing measurement validity,
- Common interests in sources of lists to measure and directly comparable data formats,
- Could measurement efforts be bundled together -- if one person is interested in one measurement initiative, they may be willing to perform others;
General Themes Across Projects:
- Tools that produces useful data,
- Publishing of the raw data within appropriate boundaries,
- Analysis and interpretation of the data,
- Education, promotion of the interpretations of the data.
- Share test helpers/collectors.
- Write docs to improve this reuse, share and do not recreate.
- A tool to generate useful data.
- People will look at the data and interpret it.
- Based on the interpretation (“translation”) of such data, the best decisions are made.
- There is more transparency on the economics of surveillance/censorship tech.
- What changes in the world?
- Reduce the amount of over-censoring.
- Shift in people’s interpretation of censorship.
- Measuring interference goes mainstream.
- Make this data more understandable
- Add crontab to deb that regularly tries to upload stored test data
- Who (user)(uid) runs cron tasks?
- Decide when to run test and submit data
- Add mechanism for feedback to user to “see if it’s working”
- Build stable .deb for Debian Wheezy
- How where to port .debs for users to install
- Auto-building OONI packages for autobuild .deb for target (wheezy)
- Document install process (and test/refine documentation)
- Set up and test collectors on M-Lab slice (test rel.)
- Build and/or retool monitoring infrastructure
- Magic to (pipeline) hook collector into M-Lab data pipeline
- Format conversations for YAMLoo to BigQuery
- Q/A check the .deb install pathway for Ooni-client
- Resolve M-Lab bouncer overhaul questions
- Run through full test pathway on 2+ M-Lab nodes, with 2+ clients (manual, test related)
- Integrate all backend code into ooni repo, which will be used to build slice package
- Integration tests for successful propagation of data through entire collection pipeline (integration related)
- Q/A test slice package
- Monitor initial client deployments for success (fail to public, fail to test, fail to parse, etc) (integration related)
- Find OONI on other milestones and answers for integration questions
- ICLab reuse/duplicate OONI code
- Share test helpers/collectors
- Write docs to improve reuse -- encourage other tests to share do not recreate tests
- Primary Questions:
- Can we have mlab-ns monitor helpers, collectors, and bouncers as separate helpers – or – can we return multiple fields?
- Can we have mlab-ns return .onions, such as in the FQDN field?
- Can we do the two above with minimal modifications to mlab-ns?
- If we modify mlab-ns, can we modify it in a way that's most likely to be most helpful to future experiments?
- How little can we modify OONI to implement the above?
- Does mlab-ns need to return a custom FQDN?
- Approach One:
- A new bouncer makes a broad query for all OONI stuff, then determines how to answer probe queries,
- Ooni: Open a ticket for making this custom bouncer -or- modify the collector to return information about the other helpers in the process.
- M-Lab: Open ticket for M-Lab, to alter the query to return JSON with all services (or all Ooni services),
- Add tickets to mlab-ooni support for the mlab-ns tweaks.
- Wheezy .deb,
- user in debian policy (where is $HOME),
- Q/A check deb installs
- determine deb repository;
- Measurement Lab Integration
- M-Lab monitoring integration,
- mlab-ns integration,
- deploy collectors/helpers on slices,
- integration test propagation of data
- Run "beta test" which allows us to do a sanity check on the data / smoke test,
- Improve integration tests to verify data is collected,
- End-to-end integration tests are an M-Lab criterion,
- Add user feedback mechanism,
- Success involves incorporating the data from oonib into the M-Lab db -- magic pipeline and format .yml within BigQuery,
- Crontab in debian policy,
- Determine when do we run the tests -- e.g. upon reboot and when in cron,
- Ensure that if network is down, it does the right thing, where the data allows us to distinguish ifconfig eth0 down from interference,
- Documentation on how to edit ``/etc/apt/sources.list`` and install the wheezy .deb,
- Monitor initial deployments,
- Integrate backend code into ooni-support package,
- Autobuild debs (talk to weasel),
- debs are already made for dependencies not in wheezy, then integrate them into weasel's autobuild;
- Wheezy .deb,
- Trial of installation OONI documentation not included, available here: https://drive.google.com/file/d/0B2q69Ncu9Fp_QlQ3UUZFOC13WGM/edit?usp=sharing
- Crontab (connect with Sam @ BISmark and OONI),
- Integrate with OONI now that “we” are using python,
- Shared scheduling,
- Python on Android,
- Visualization and dashboard,
- Test helpers (OONI),
- Common data pipeline (ICLab), data format and transport,
- Bridging between technical and advocacy/policy/impact:
- (Short term) Having measurements running on certain devices in X countries,
- Reproducible standard method,
- Agility of measurement:
- When, where, what, how,
- Stable base-test,
- Make information available for advocacy:
- No more one-off, paper projects,
- Archiving Data.
- Get WA + Peny Fns Working
- Load Sathyas’ code on rPi
- Get web UI + measurements on same rPi
- Perform Userbase study for deployment
- Determine what we can steal from OONI now that (we are using) Python
- Get rPi sending data back to data server for analysis
- Get rPi’s communicating with controller receiving experiments/comments
- Get tests that the lab is already running running daily on Pis (synced HTTP tests, DNS, traceroute)
- create a scheduler that makes cronjobs
- automate informed consent procedure (risk: levels, link consent and risk level to the rPi)
- Get experiments running for Indonesian election
- Look into test helper servers/overlap with OONI
- Figure out and implement a sandboxing technique
- Have a debian package for background tests
- Multi Platform support Linux (Mac?)
- Multi-platform support: Android, iOS
- Web interface for researchers
- Multi-platform support: Windows
- Integrate the differentiation detection App into the platform
- Wants use documentation from OONI,
- Willing to audit usability from a user’s perspective.
- Find partners to run code bundle probes together
- Common analysis
- Visualization and dashboard
- Pipeline and data format
- Python collaboration
- Common data pipeline (ICLab), data format and transport,
- Goal: thousands of clients per day performing measurements in dozens of interesting countries.
- Collaboration on lists of things to measure (ICLab, etc.)
- Data Storage (M-Lab, etc.)
- Visualization and dashboard,
- Talk to Zooko about service monitoring deployment
- Explore issues blocking encore from doing service monitoring
- Develop per-country measurement lists (do we know they are safe?)
- Build more crawlers (and figure out whether we need them)
- UI for showing sites we measure (targets)
- Do a PR push
- Write a blog post
- Send encore data to M-Lab for store
- Recruit more professors,
- Get Slashdotted,
- Brand as service monitoring,
- Buy some ads.
My Speed Test
- Python on Android
- Sharing test cases with OONI and ICLab
- Common data pipeline (with ICLab)
- Common scheduling language (with ICLab)
- Push newest fixed version (minus beta features)
- PR pushes
- Improve DNS consistency test by using reverse lookup
- Figure out (Kivy) python on Android
- Collect DNS resolver used
- Figure out transport for rsync on Android
- Figure out what tests exist
- Common scheduler w/ Raspberry Pi
- Integrate MST to a common data pipeline with ICL
- Implement HTTP header manipulation test
- Notification for real time monitoring
- Storage on M-Lab
- Common data format
- Correct IPS for DNS queries to check for spoofing v. dual homing/anycasting
- Analysis of what a correct DNS response is,
- Find keyword dictionary for keyword blocking test.
- Decide correct log/data formats especially for high bandwidth tests.
- Determine most accessible data storage location (S3 v. M-Lab v …).
- HTTP keyword blocking test,
- Analysis of data to make comparable to existing reports.
- Automate and package data collection program,
- Create reference data analysis program.
- Release data and Code.
- Get people to contribute tests and use data.
- find partners to run code
- Encore data on M-Lab and host crawlers
- Dashboard for results over time
- Share code with our work on browser extension
- Satellite data on M-Lab
- ICLab /OON/ helper integration help
- When the platform is well-documented and built for others to use for censorship research.
- Hosting compelling experiments that produce data that anyone can use.
- Produces verifiable & repeatable/reproducible data.
- Live data
- Catalyze collective action
- Be the host for resources that provide definitive censorship “status”
- Bundle resources to make sum > parts
- Global dashboard
- enabling civil-society to act on the data generated by the platform
- exposing censorship ⇒ reduction in global censorship and in acceptance of censorship
- define audience(s)
- what is possible?
- API for data access --->
- API for experiments ---> appropriate to audience
- new experiments/automation of deployment
- adoption (interaction w/researchers, experiment writers, gov’t, CSGs, users) = awareness
- of stance on neutrality
- competitiveness among researchers
- end-user visualization
- don’t overly reimplement
- from many people (researchers, commercial service providers, users)