Discriminator — Grant Report # 2

Brett Gaylor — Mon, 23 Aug 2021 18:22:30 +0000

Project Update

The Discriminator web documentary that Grant For The Web supported was completed in early June. Our team produced an interactive experience roughly 15 minutes long that explored my experience of having my photos included in the MegaFace database, and the phenomenon of facial recognition databases more broadly.

The web documentary was built as a React web app, which was web monetized through Coil using the web monetization API.

The music for the project was composed by David Drury and released under a Creative Commons license on the Free Music Archive.

Progress on objectives

Goal 1 - "incite curiosity and educate users to the social concerns surrounding facial recognition, AI and disinformation"

This goal has been completed and feels successful. The key deliverable for this objective is a publicly available web documentary, Discriminator, which premiered in June at the 2021 Tribeca Film Festival. It has been well received and reviewed, and found a mass audience at one of the most prestigious film festivals that curates interactive media. With its focus on how photos end up in facial recognition databases without consent, the project makes an original contribution to the field.

Looking back, focusing our grant activities on this as the central objective was key to having a successful project. Web monetization, as I’ll get into, was a part of our experiment - but focusing our energy on creating a compelling piece of filmmaking allowed us to control our own destiny while also leaving room for experimentation.

Goal 2 - create a popular and professionally produced documentary that other creators can study and emulate

This goal spoke to creating a reference implementation of a web-monetized web documentary, given that at the time of our application there were no models to follow.

Developer Jae Perris implemented the web monetization API within a React web application, publishing the source code on Github. Jae also published a simple blog post outlining the development process of the documentary - including both the machine learning work to execute the creative as well as the web monetization work.

In truth, the web monetization development work was fairly straightforward - our experience was that barriers to success for micropayments of this sort are not technical, but rather lie in finding a large enough user base. Nonetheless, for others wishing to create a web documentary that takes advantage of web monetization, they now have an open source project to build from.

Goal 3 - experiment with new ways that we could finance our work and establish more direct relationships with our audiences

As mentioned in our interim blog post, our strategy shifted from financing the work through web monetization to having funds generated go towards advocacy work in the field. We were successful in partnering with the Surveillance Oversight Technology Project (S.T.O.P.), and every web monetized visit to Discriminator is channeled towards their 501(c)3.

This was the most challenging aspect of the project. While it was a simple matter to implement the web monetization API, it was exceedingly difficult for STOP to register for a payment pointer. The only crypto-wallet that accepts Interledger payments, Uphold, has stringent requirements for 501c3, charity and corporate accounts. Despite STOP dutifully jumping through their hoops, it took months to have a payment pointer generated.

Once we did have their payment pointer, the next burden became the reality of the user base of Coil subscribers. Even though our project premiered at a top tier festival and received great press on launch, the traffic that was generated by web-monetized users was simply too small to generate any revenue. In total, web monetized views garnered $6.95.

In our case, this did not prevent the project from release: we were grant funded from multiple sources. It did, however, raise a key challenge for the web monetization community: if the goal is to encourage creators to publish outside of the major platforms, the challenge of raising an audience several orders of magnitude higher than we were able falls to each individual. For web monetized projects to use their viewing revenue as part of their project financing, then, they must partner with existing brands or partners that can bring large pre-existing audiences.

Key activities

The key activities during our final reporting period were research, web development, music composing, animation and communications.

For research we partnered with Adam Harvey. Adamis an applied artist and technical researcher based in Berlin focused on computer vision, privacy, and surveillance. He is a graduate of the Interactive Telecommunications Program at New York University (2010) and is the creator of the VFRAME.io computer vision project, Exposing.ai dataset project, and CV Dazzle computer vision camouflage concept.

Collaborating with Adam, I was able to trace how my photos had been included in a variety of databases. This formed the basis of the data visualizations that make up the second chapter of the documentary.

For web development, Jae Perris and Hang Do Thi Duc collaborated to produce the interactions, computer vision and AI that makes the project interactive. This was where the majority of our time and funds were spent. The project repo on Github is available for any developer to fork, modify and inspect.

Jae Perris also wrote a detailed readme as part of the repo, including the web monetization implementation, which can be read here.

The original score was composed by David Drury, and was released under a Creative Commons Non-Commercial Attribution license. It can be downloaded directly from Soundcloud here.

The Animation was created by Darren Pasemko and Kent Hugo using a variety of techniques including Generative Adversarial Networks (GAN). We were supported in this by RunwayML, a software project to make AI models more accessible to graphic artists. The Runway team gave us free computing time on their GPUs to train the models using photos of me.

Communications and marketing

I contracted GATPR to assist in our press releases and public relations. This was a somewhat difficult assignment for the agency as interactive web documentaries are odd ducks - not reviewed by most critics and not understood by most viewers.

Despite this, the film had several dozen reviews, mentions, vlogs and podcasts. Some highlights:

Indiewire:
https://www.indiewire.com/2021/06/tribeca-immersive-2021-program-free-1234643259/
“Over the course of three remarkable chapters, Gaylor tracks the history of photo-sharing sites like Flickr, and how corporations ultimately exploited Creative Commons licenses to access personal data and put millions of people’s lives online without their permission. If you haven’t heard of MegaFace, the publicly available facial recognition platform, Gaylor will make sure it haunts your nightmares. In the process, he’ll also deliver a searing breakdown of modern technology’s most perilous extremes, and what it takes to protect your privacy in the digital age.”

Amber Mac:
https://ambermac.com/thefeed-discriminator-doc-zebra-loop-returns/

Amber Mac

@ambermac

On this clip of #TheFeed, director Brett Gaylor @remixmanifesto discusses the sinister side of facial recognition technology and his new documentary, Discriminator.

This episode of #TheFeed is powered by @TPLink.

To hear the full episode, visit AmberMac.com.

12:38 PM - 04 Aug 2021

No Proscenium:
https://noproscenium.com/tribeca-immersive-diary-2021-updated-6-9-21-9c5724706052
While the message is a dire one the messenger is a delight. This work puts the interactive, data-driven journalism projects of big media companies that win huge awards to shame. You’ll learn oodles. You’ll be delighted and horrified by the tech in equal measures. You’ll start contemplating how long it would take you to learn how to code a virus that would shut down the internet forever and ever.

WPBS Studios: Inside the Stories

What’s next?

The next stop for the film will hopefully be an international premiere at the International Documentary Festival of Amsterdam this November. I have applied and hope to hear back in the fall.

The project will also be evolved as part of my PhD thesis at Simon Fraser University. I plan to contact more individuals who, like myself, were included in facial recognition databases without consent. Interviews with these individuals will form the basis of a podcast and a research paper that explores the particulars of harm that are associated with being included in large databases.

We also plan to host an online screening and Q&A in the fall as a fundraiser with S.T.O.P. I'm hoping that this could emerge as a way to integrate web documentaries more effectively with the micro-payment model, particularly if we can coordinate with Interledger and Coil to promote it within community channels.

What community support would benefit your project?

The project still needs more people to watch and share it! I hope that it’s entertaining and educational for folks, and that it inspires others to make work on the web. There is still so much to do outside the platforms!

Please keep an eye out for our community screening and come join us!

Additional comments

I’m excited to continue experimenting with web monetization and other forms of decentralized financing for creative work. I think there is promise in combining this approach in partnership with a larger brand that is still publishing on the open web.

Relevant links/resources (optional)

Film:
www.discriminator.film
Github:
https://github.com/Honga1/discriminator/
Soundtrack:
https://soundcloud.com/brettgaylor/sets/discriminator-the-score

Discriminator — Grant Report #1

Brett Gaylor — Tue, 02 Feb 2021 23:52:40 +0000

Project Update

Discriminator is well into production, with ongoing work in research, animation, music and web development.

The initial phase of the project was in the creation of a set of prototypes to explore the possibilities of using facial recognition within a web documentary. This prototype explored the possibilities of the "wav2lip" project, which modifies images and video files by synchronizing their mouth movements based on a submitted audio file. The goal was to present a "deepfaked" video by seamlessly integrating it with static video files. The results can be seen here:
https://three-videos.jaeperris.com/
(Google Chrome recommended)

In parallel, an extensive research phase was undertaken to understand how the author's flickr photos were used in facial recognition databases. In partnership with researcher Adam Harvey, we explored how the Yahoo Flickr Creative Commons 100 million database (YFCC100M) became the basis of several large facial recognition databases including MegaFace. This research modified and focused the direction of the project - rather than exploring disinformation and synthetic media, the project now more specifically focuses on the problematic use of CC licensed images in facial recognition databases. This is explored more full on Adam Harvey's website, https://exposing.ai/

Building on these foundations, we are now moving forward on animation with Darren Pasemko, music with composer David Drury, AI and web development with Jae Perris, and design and creative technology with Hang Do Thi Duc

Progress on objectives

Goal 1 - "incite curiosity and educate users to the social concerns surrounding facial recognition, AI and disinformation"

This is our central goal, and the documentary is designed to educate while being entertaining. As research has progressed, however, it became clear that the primary dilemma was the unanticipated uses of creative commons photos. Like many others, I (Brett Gaylor) uploaded many photos to Flickr under a Creative Commons license only to discover many years later that they had been included in large facial recognition databases such as Megaface as a result of licensing choices. The goal of the documentary, therefore, is to widen understanding and conversation about the unintended consequences of openly sharing data.

Goal 2 - create a popular and professionally produced documentary that other creators can study and emulate - this is going well, and we are releasing all of our code on Github

Goal 3 - experiment with new ways that we could finance our work and establish more direct relationships with our audiences

As the focus of the film has tightened, I have become more aware of advocacy groups in this space. For this reason, I have decided to focus the web monetization component of the project on raising funds for groups pushing back against surveillance systems. I am in talks with the Surveillance Technology Oversight Project.

Key activities

Our key activities are research, web development, composing and animation.

You can see progress in all of these areas in this prototype
https://three-videos.jaeperris.com/
(Google Chrome recommended)

Communications and marketing

Our project was recently featured in The New York Times
https://www.nytimes.com/2021/01/31/technology/facial-recognition-photo-tool.html

What’s next?

We are gunning for the finish line! We aim to have music composed by the end of february, animation completed in early March, and web development in April.

What community support would benefit your project?

What we will likely need the most is word of mouth promotion.

The Interledger Community 🌱: Brett Gaylor

Discriminator — Grant Report # 2

Project Update

Progress on objectives

Key activities

Communications and marketing

What’s next?

What community support would benefit your project?

Additional comments

Relevant links/resources (optional)

Discriminator — Grant Report #1

Project Update

Progress on objectives

Key activities

Communications and marketing

What’s next?

What community support would benefit your project?