Crowd the Tap: People Protecting Our Tap Water

The Full Case Study

Crowd the Tap: People Protecting Our Tap Water

Project Description

Crowd the Tap is a citizen science project relying on people sharing information on home plumbing and service lines to produce the first robust national inventory of water pipe materials to inform lead pipe replacement projects.

There are currently no reasonable estimates of how many homes have lead pipes. Although the installation of lead pipes was banned in the US in 1986, much of the US drinking water infrastructure still contains lead pipes. In the past, national tap water infrastructure assessments have involved researchers requesting information from water providers. Unfortunately, these efforts produced only a partial picture of water systems with lead service lines.

Crowd the Tap is the first U.S. Environmental Protection Agency (EPA) funded project that promotes access to safe drinking water by empowering individuals to investigate their piping infrastructure (steel, copper, plastic, or lead). Once data is collected, the research team focuses explicitly on lead pipes, a known lead source in tap water, hazardous to human health.

This case study was prepared by Alycia Crall (Director of Community at The Carpentries) and Tiana Curry (ADSA). They interviewed Caren Cooper (NC State University), the project’s principal investigator. The case study is presented here in the form of an edited interview.

Resource: Crowd the Tap: Empowering Communities to Examine Their Lead Exposure

Lifecycle stage 1: Problem identification

Lens 1: Positionality

Alycia Crall (AC) and Tiana Curry (TC): Given your position with this project, how much agency would you say you have to redefine the problem? 

Caren Cooper (CC): There is not much funding for this project, so I can redefine it if needed. The project has already evolved. It started as an inventory of pipes and then switched to building and verifying a risk predicting model. The next phase will be validating a field method. We keep adapting the work and the resources.

AC & TC:  Redefining the research question is easier because of the type of funding or because you are not too dependent on the financing.

CC: I think a little of both. The funding, the team of collaborators, the projects that emerged from that research team (the Lemon test see below), and the need for more data for the model. We started building a model because of the gaps in the rest of the project. It is a natural evolution.

AC & TC: Do you have the most relevant expertise to address and frame this problem? Who else might have the expertise, and who is considered an appropriate expert?

CC: In many ways, I do not have the relevant expertise. Policy specialists, communities, and educators affected by lead in water have much pertinent expertise to understand how utilities work and how local state governments are involved in lead service line replacement. I believe the solutions are more policy-oriented: comparing water policies is a giant leap. Scientists have been trained to hand their work off to someone who understands the policy:  if we separate these two issues, it is hard to be relevant. So, I have been trying to learn more.

AC & TC: Does the project plan incorporate regular checks, discussions, and documentation about the ethical dimensions of the project? 

CC: We had a project plan: I honestly do not have good documentation about the ethical dimensions of the project. I have had many discussions with data ethics experts to create a consent document. The content is good, yet the document is overbearing, and I wish it had a graphic designer who could make it easier to read. I am unsure about regular checks: since the project keeps evolving, we have many unplanned reviews. At every step, new considerations and discussions emerge about specific ethical dimensions. Being clear helps with trust, which is essential for a project about water: there are so many trust issues.

AC & TC: Do you feel you are being sufficiently transparent with the people involved?

CC: The consent form lays out all the relevant aspects of the project: how the data are collected and analyzed, who can see them, what is private, what is not, what are the options for sharing data, etc. As mentioned, I wish we had communicated it graphically. We are reaching a limit in terms of user-centric design. Consent forms should not be like terms of use that nobody reads. They are supposed to be clear so that anyone can understand: they are a tool for transparency.

Lens 2: Power

AC & TC: Who are the stakeholders? How broadly are they understood, and to what degree are they included in the problem framing process and the research or project overall?

CC: After the 2014 Flint water crisis, funding agencies such as the US Environmental Protection Agency (EPA) labeled lead in drinking water a national priority. The EPA needed to find where lead is ahead of time to replace the pipes before something terrible happens. The stakeholders are people exposed to lead in drinking water: we do not know who they are or where they are. We know there is still lead infrastructure, and people are at risk from it. Stakeholders are people who represent those interests and who have been part of the framing process. I tried to get more input from the Lead Service Line Replacement Collaborative, a consortium of NGOs dedicated to protecting drinking water, including the Environmental Defense Fund. I could have been better at getting their feedback; they also tried to get more input from the utility companies. As for me, I do not have connections with utility companies, and I have no experience understanding their perspective: I might have blundered into it with some early missteps. They are important stakeholders, and it is a project failure that they were not engaged sufficiently in the process. Many have hesitancy about partnering with academics and projects they do not have complete control over. They worry that they might get blamed. The point of the project was not about the utility side – ” let us leave it to utilities to figure out what pipes they have and deal with them” – but was consumer-centric: “we are going to help consumers who do not know that utilities are responsible for the lead in water.” We were trying to support this notion that it was a joint responsibility, which is how it is considered in law and what the utilities should like. It is also critical to note that there are a lot of different utility perspectives; some are very large and run by city governments, and some are private and minimal. In North Carolina, which is just serving residences, we have counted over 2000 utility companies.

AC & TC: Who has the responsibility for framing the question?

CC: The EPA first framed the question after the Flint water crisis. EPA framing was top-down in response to public concern, and the research team I am part of proposed investigating a more consumer-centric model.

AC & TC: Has your research question undergone “transformations” or “translations” in moving from one system/organization to another? Who has the final “say” in defining the research question?

CC: In terms of defining the research question, the project is centralized to Principal Investigators’, like my colleagues and me: there is no particular mechanism to make a more democratically distributed or equitable decision process. We are responsive to the realities we face with different stakeholders as much as possible.

AC & TC: What are the power dimensions identifiable in your project? How does it influence the national issue of clean water?

CC: We do not have enough data about lead service line replacements. We did not generate enough interest to move the needle far on that. I am more optimistic about the Lemon test, the last phase of the project, which will be delivering a product. For a $15 off-the-shelf kit, anybody can have a trusted way of testing lead in water in their house without having to send it to a lab. Not having to rely on a lab or an external source is science power in the hands of the public. It creates evidence that people can use to be watchdogs, that they can trust and double-check. That is what I am hoping for.

AC & TC: What happens when the project ends? How do you exit in a way that does not leave a community lacking or missing a resource? How does that transition happen? 

CC: The initial framing considered that residents could not test their water. For the project’s second phase, we built a website that runs a model to collect simple household information and analyze easy chemistry strips. We are still trying to build resources and capacity, so people can know if they have a problem with water without having to send it to a lab or trust their utility. When the research project is done, the resources will be available, including the online citizen science hub SciStarter website, the model, the Lemon test protocol, and all the videos.

Lens 4: Narrative

AC & TC: Is your research framed as a question of justice or a contribution to the public good? How does the framing refer to a particular conception or theory of justice or public good?

CC: I interpret justice as something related to fairness since it is an environmental concern, the right to equitable environmental benefits, risk, and burdens. I interpret public good as encompassing different ways that science and civic outcomes can benefit some segments of society at least. Equitable, as opposed to equal: the people receiving a lot of the burdens and risks are at least also receiving the benefits. Moreover, it starts with water because there are inequities in water systems at different spatial scales based on race and income. A critical dimension of the research is to examine whether there are injustices. We recognize that disparities in water systems exist, and we investigate the racial or economic inequality in the risk of exposure, which is a justice issue. The other critical component of this research focuses on drinking water infrastructure, the general identification of lead service lines, and home plumbing with the idea that it could be replaced or mitigated, which is the public-good dimension. We have a substantial justice and public-good-oriented approach. However, the justice part only goes so far as to identify if there is an injustice. It does not have a particular framing to solutions.

Lifecycle stage 2: Data Discovery

Lens 1: Positionality.

AC & TC: Who collected the data? For what purpose? And how did they understand what they were doing?

CC: The data were collected by anyone who volunteered their time. Some were students in college or high school. Some did it through their work. Verizon has developed a program for their employees to participate in our project. Others did it because they heard about it on NPR’s (National Public Radio) popular Science Friday show.

Most people learned about the project from promotional material they read – an article, a radio show, a teacher, or a public library. And the website provides videos: we have four videos, one about the project in general and three about how to do specific project components. For volunteers, the primary purpose was to share information about their plumbing materials for improvement and for the project to know what is out there. A subset of participants has also used our chemistry strips and collected water to send to the lab. Different participants engage in different ways: the ones who participate with the chemistry strips and the water collection are helping build a model to estimate the risk of lead in water. Individual contributors enable us to get a better sample of what is out there, and people using the kits are helping build a model about the risk of lead in water.

AC & TC: What kinds of identities are captured in the data?

CC: Questions are about identities in their household. In a given home, the survey asks about ethnicity and income. Is income an identity? We also ask about people of childbearing age, infants, children under five, and the city, state, and zip code where people live. I Am trying to think if I can link this information to a person: I am probably able to ask SciStarter to tell me how participants got to the project (Library, School, NPR, etc.).

AC & TC: What gaps are in the data? In particular, what identities are not collected? And I would probably include: why are specific identities not contained in the data?

CC: We do not collect demographics about individuals, just about houses. We focus on the minimum number needed to understand racial and economic disparities, with a particular interest in vulnerable populations – infants and children. We do not know if the household is a family, roommates, or the living situation. We do not ask if they own the house or not. Likewise, we do not capture their citizenship status, either. If four people and four different races live in the house, they will click four races. Now that I think about it, I should not have done it that way: it could be confusing. When we were thinking about racial and economic disparities, we were not about migrant communities or people with disabilities. So even though we had that justice lens, it was not part of the research question.

AC & TC: Where do you feel the gaps were in the data? 

CC: The most significant gap is that the demographics are not by an individual but at the household level. And everything that we are doing, like even the risk, the model of risk of exposure, is at the household level.

Lens 2: power.

AC & TC: How was the data collected? What restrictions are there on access to the data? 

CC: It is a citizen science project:  we call it to report data, like a survey through a website. Users need the Internet. It is housed on SciStarter.org, so participants do have to be account holders on SciStarter to enter their data. I have not yet created a good metadata document to support the questionnaires; this is on my to-do list. The websites and all information related to the project are in English and the file downloads as a CVS in the Word Office Suite. In terms of public access, if someone is not computer savvy, they need to have the data digested and a summary.

AC & TC: How have you considered improving accessibility? 

CC: One is the creation of a model: when people enter their data, we can give them an estimate of their risk. For those who have done the lab tests, we are working on getting them to report: we looked at different styles, what to write, how to present it, how we classify risk as low, medium, or high, and what recommendations go with that level of risk. When people choose to participate, their data is anonymized, and this anonymized data is available to anyone: this is in the consent form. Participants can decide to separate their names from their data if they do not want their names public. Names are public so that we can individually thank participants, for instance. And then, they can also decide whether their address is linked to specific data: high-resolution data can be shared with other groups like the U.S. Environmental Protection Agency, their water utility, a state agency, or BlueConduit, a group that’s trying to estimate lead risk in service lines.

AC & TC: What enables you to access this data and not others? 

CC: SciStarter’s database manager keeps the data on a secure server, and I have access to download it with personal information. I can access the full version, not the anonymized version, and I store that on my laptop. If my state agency or utilities wants the data, I will give data from anyone who checks that box and consented. I have not looked at how many people have accepted being recognized: on average, about a third of people opt to have their name public, which surprised me. Most prefer not to have recognition, or maybe they feel some risk involved with disclosing that they have done the project.

AC & TC: Is the data valuable to the project? 

CC: The data do have problems. It is not like a representative sample of people; it is reaching people who have listened to NPR or have enrolled in North Carolina State. But within those constraints, it is beneficial. It provides insights into potential disparities, and it helps build the model. Maybe the state will not commit lead service line replacements based on what homeowners say; they will want to go out and validate it for themselves. There might be some instances where people will not view the data quality as trustworthy enough because it is citizen science, but I consider it good. That said, I do not trust students’ data: I had students cheat on a citizen science assignment, and I have learned how to make that not happen: if a student waits and realizes, “Oh, I cannot do this at the last minute,” they are going to cheat because they have got to get it done.

AC & TC: Who do you think the data would not be helpful for or valuable to? 

CC: State agencies find the data useful to consolidate their assessment and request specific federal government funding. When distributing that money, I do not know if these agencies will trust the data to guide them. If there were a lawsuit, it probably would not hold up in court, whereas it will when we publish our papers in the peer review journals. That is how it is for most citizen science data: it is so hard to publish.

AC & TC: Do you know this data’s intended versus unintended use? 

CC: The intended use is to improve drinking water infrastructure and safety; an unintended use could be companies using it to target to whom they want to sell bottled water. The intention is to find solutions to ensure everyone has access to safe water, not to create a client database for certain companies. Other unintended usages could be a child custody dispute and lower property values.

AC & TC: What population does the data cover? 

CC: It is opportunistic, haphazard sampling – as opposed to ecology, where we do a stratified or random sample – because self-selected volunteers or people contribute data to the project. It is not representative, but it is just a constraint: I cannot conceive of a way to make it representative. The closest we could get would be if every utility were willing to send information to their customers about the project and invite them to do it. We are overrepresented in North Carolina because of the partners we have had.

AC & TC: What years do your data cover? What has changed?

CC: We launched in 2018 and have just been going slowly. The big switch was to do the model validation and add the water chemistry and the water collection; next, we will add the Lemon module. The question and the data type have changed a bit throughout the study. We had started with libraries across the country, but soon after, there was the pandemic, and most of the library programming closed down; we then relied a lot on school programming.

AC & TC: What is the geographic level of the data?

CC: It is open. Anyone in the US is invited to participate. We have had some people in other countries that have wanted to participate through the Verizon program, it is just not completely set up, and I do not know enough about water systems in other countries.

Lens 3: Sociotechnical systems

AC & TC: Where did the data come from? And why was the data collected? 

What systems, search engines, databases, archives, libraries, and social networks are you using to find data sources? 

CC: We do not collect any other data sources. It is just volunteer-generated data. I find people through libraries and networks, and schools.

AC & TC: How is the data curated?

CC: It is currently on a secure server that SciStarter runs. At some point, I’ll give it a creative commons license and put it in a public archive.

AC & TC: What classification systems were selected to collect the data? What types of collecting and recording strategies and techniques were used? And for what organizational purposes?

CC: Curating is creating the metadata. For classification systems, we used a Web form on which people uploaded photos of their chemistry strips for validation. They were standardized photos with a color chart. I could not find metadata standards about plumbing and service lines. For our demographic, we choose standard classification systems.

Lens 4: Narrative

AC & TC: What conceptions of justice informed the data collection practice?

CC: There are two parts to it. The first is participatory justice: everyone can be involved because it is a citizen science project. Another aspect is the notion of distributional justice, recognizing that the distribution of the risk of lead pipes would vary with race and economics.

AC & TC: Would you say that your efforts around privacy or information protection were a part of this?

CC: The data collection practice involves people sharing their names and address, the street address is shared beyond the research team. The project grew to have options where people can control how specific data is used and could share data with different groups: state regulatory agencies, utility, and other research groups.

For a while, we have had different community engagement specialists working with diverse communities to overcome barriers to participation. The most important question was: what are the benefits of participating? People were asking, “Why bother to share whether I have lead pipes or not when it only seems like it only provides more risks than benefits?”
We have identified a flaw: the project is based on individual consent, there is no community consent. So I might say, “I want, I do not want anything disclosed about me or my neighborhood,” yet I cannot control that. And my neighbor might say, “Well, I am letting everybody know we got lead here.” We did not have a way to make that happen at a community level.

AC & TC: Has this influenced how you engage with the community? Are you trying to figure out ways to protect the community?

CC: The whole point of the research is to find where lead pipes are, so the utilities can better manage water corrosion or replace pipes and look at the neighborhoods’ demographics. There is no one community; it is anyone who wants to contribute. That is why it is hard to provide anonymity for a community as a whole, given that we share the data at the zip code level and ask people to give the name of their utility. The most we can do is protect individual privacy.

I believe the risk could affect financial/real-estate values because residents do not have to disclose lead issues. Still, when it becomes public information, the risk is reputational and could be considered a stigma of lead or potential lead exposures. In Flint, children have been removed from homes because of unsafe conditions: it is extreme, but it happened. The benefits are attention and public and government concern for fixing the problem. It is much harder to fix if it is never brought to light. So those are the pros and cons.

AC & TC: What data types are needed to answer this question? How does envisioned usage of your research conclusions impact the data collection?

CC: We ask about household demographics since it is easier to anonymize location: we do not have to look at census maps. The lead risk is on the house residents; we do not have to reveal anything about them. We are still asking what utility they are connected to if we need to start parsing our sample by utilities to increase our research impacts.

AC & TC: What are the expected research impacts?

CC: To document racial and economic disparities in lead risk. This is why we asked questions explicitly at the household level about race and economics, so we do not have to do it at a census district or neighborhood level.

AC & TC: In the context of your research, what are all possible data sources that can be collected? 

CC: We rely on our website, Crowdthetap.org, and the other relevant data sets from the Environmental Protection Agency (EPA). We have additional data from the utilities and state agencies who are starting to comply with new regulations making data collection mandatory from utilities about the types of pipes. It is still utility-focused for the most part. We might use the census a little, but we are trying to get the household data.

AC & TC: Do any of the visible relationships lend themselves to arguments that might be potentially harmful to a particular community?

CC: People are handing over data to me, and I am sharing it with others, state agencies, federal agencies, utilities, and other researchers: how this data is used could be harmful. The participants might not have direct relationships with those groups. We used to have this step in the project in which participants: after reporting on their home pipes, they had to call or write to their utility to see what could be the utility response. We took that part out because it was too complex. But it made visible that there might be no relationship between those who contribute data and those using it. The data-use agreement does not restrict what people can do with the data.

AC & TC: Are any harmful arguments that can be built from assuming a connection between the community and utility?

CC: The utility could think, “All these customers are saying they might have this problem. We are going to need to raise rates. We know we are not going to get any federal aid, so we are just going to raise rates” I do not know… There are other industries known to intimidate watchdogs who monitor them. Maybe I am naive, but I do not think that is the case regarding water utilities.

AC & TC: Where do your assumptions come from? Were they correct? 

CC: We assumed it would be hard to reach beyond the typical citizen scientist participant in these top-down projects – and that has been the case. The assumptions come from the literature on citizen science participation and disparities in drinking water. I recently looked at the participants’ demographics: 11% of our sample is Hispanic, and 16% of the US population is Hispanic. 1% of our contributors are indigenous, and 1.6% of the US population is indigenous. For African Americans, we were under-sampling. About 10% of our participants are African American, which is below the national average and particularly low for North Carolina, where 22% of the population is African American. We want to get a more representative sample and not just sample white, affluent places.

Even with our small sample size, it is starting to look like there are disparities in the distribution of household lead pipes. For now, the sample is too small to know for sure: we need more samples because I suspect it will be the story to tell.

We also assumed that people might perceive it as risky and have trust issues. And that seems to be the case. In many citizen sciences projects, there is no consideration about hiding people’s names. It is a big thing to recognize the volunteers publicly. We gave people a choice, and it has been pretty steady: about a third choose to be public, and the majority have opted to stay private. Participants do not want their names mentioned, even their demographics. They do not know to whom they are giving data. There is no trust because there is no personal relationship. There are perceived risks when it comes to disclosing information about these questions.

AC & TC: Is the data not being put into the model? What are you using it for?

CC: For a different model. It will be used to analyze racial and economic disparities in water quality attributes and the risk of lead. Some data are used for this analysis and are not included in the model. And by model, I mean a Bayesian Belief Model (BBM) or a network model: it helps predict if a household has lead. That particular model is different from the analysis I will be doing.

AC & TC: How was that decision made about the data used for the different analyses you are doing?

CC: The models have been built on an earlier model for wells. And then the other data for my analyses are based on literature, scientific papers, etc.

AC & TC: How are the variables of interest framed? What are the narratives that these patterns suggest? 

CC: This is an exploratory analysis. I do not know what it means to say how it is framed. I could answer after I have done the analysis. Emily Butland and her lab, my collaborators at North Carolina State, are doing the Bayesian Belief Network and the AI, and I will be doing regression models. The Bayesian Belief Network predicts the risk of lead and water at the household level. Data come from pictures of chemistry strips, the standardized photo the participants submit. Grad students train the model to process those photos, fed into the BBN. The rest is just regressions that I do related to race and economics with different water quality parameters. I use Excel and SAS.

AC & TC: Are there any known concerns or limitations to using this analytical tool on similar data sets?

CC: It is proprietary. In terms of replications, some people cannot replicate it because you have to have a license.

Lifecycle 3: Use of Analytical Tools

AC & TC: What do false positive or false negative results mean in the context of this research? 

CC: The Bayesian belief model is a decision tree model. Right now, the current version does have an unsettling amount of false negatives and false positives. It will get better over time. Until then, it is hard to communicate the evolution process to participants. It is not like a pregnancy test. We keep comparing our model to pregnancy tests, where people just want to know, “Am I pregnant or not?” We wish our model could be so good and that we did not even have to talk about false negatives and false positives, but we are not there yet. The researchers doing the model know how to handle it and interpret the results. The public wants to take action based on our results: for this project, a false positive would mean that someone’s household might think they are being exposed to lead, but they are not. So they might take action, get filters, or do whatever they do not need to. A false negative, in this case, would mean that they have lead. And that is worse.

AC & TC: Do you know the chances?

CC: We have estimates. Are they helping people? I am excited about the Lemon test because it is more definitive.

AC & TC: How would it affect them if implemented in predictive technology? 

CC: The participant might take actions that are not necessary on the one hand, or they might not take steps when they really should be. It is a false sense of security that keeps them at risk.

AC & TC: What actions do the results recommend and to whom?

CC: If there is any chance that people have lead in water, a whole set of options can include: 1. certified filters for removing lead from water used for human consumption; 2. flushing the system for five minutes before consuming the water is only feasible in places where there is much water, or it is cheap; 3. getting testing from a lab, either one’s utility or an independent lab. Then there are other levels of investment: household filters, service line replacements, and plumbing replacements. You can also work on community organizing and petition the government. There is an entire suite of actions, from easy to complex.

At Crowd the Tap, the hope was that people might self-organize. The forum does not have many users with lead problems. That said, most participants are located in North Carolina, which has lead, yet it is not at the forefront of people’s issues as much as it can be in other states.

AC & TC: What could help the public better understand the results of this project? What is the predictive model saying? 

CC: Improve scientific communication. Instead of people just entering data and then getting a result from the model, I wanted them to be able to change parameters and see what affected risk, to get a sense of how it worked and what it was sensitive to. It would demystify the model, but it is not yet happening. People hate models. They hate climate models. People do not like any of those things that are not certain, and they always use them as an example of science not working. If we could solve that problem, we would be doing pretty well.

Sociotechnical systems

AC & TC: Do the analytical model chosen to address the research questions, and the intended use of the data support the research hypotheses? Are the assumptions of the analytical model met reasonably well in the context of the data situation?

CC: Can a Bayesian model be called an analytical model? I guess it is. I am thinking, “Well, do we have research hypotheses?” We discussed this because it is very exploratory, and the analytical model is not used related to hypotheses about race. Questions about whether there were racial or economic disparities will be treated as a regression model.

AC & TC: Do the data coverage or representatives meet the needs of the models? Are the data quality needs of the model met?

CC: We currently need more samples from areas with higher lead levels. In our samples from North Carolina, I think we had only one sample exceeding U.S. Environmental Protection Agency levels. There were many samples with little traces of lead, tiny amounts: the EPA level was only surpassed in one sample, the level of 15 parts per billion. I bet it will be a model with lower false positives or negatives when we get samples from areas with much higher lead levels in water. I am thinking of that in terms of representativeness. Currently, the water systems we are sampling from are relatively safe. We cannot build a model that identifies all the risk factors without sampling in some systems with much lead, so it is not representative. Plus, data were collected mainly in North Carolina: we hope to get teachers from across the country to help diffuse the project beyond.

AC & TC: Are there benchmarks to compare the results, i.e., the descriptive statistics or results from other studies? What can be learned from the fitness-for-use or benchmarking assessment?

CC: We will compare our results with other studies of water systems. Studies have estimated that around 7 million lead service lines are still nationwide. I do not know what other benchmarks we might use. I cannot think of other studies that could relate specifically to this that would be super useful.