Automating Data Collection and Analysis for Solar Energy Initiatives
Semester: Spring 2025
Praxis Course: DSCI 310: Data in Action
Faculty Advisor: Jennifer Spohrer
Field Site: Philadelphia Solar Energy Association (PSEA)
Field Supervisor: Liz Robinson, Rob Celentano
Praxis Poster:
DSCI_Jenny_Le_RevisedPraxisPoster
Further Context:
During our time in the Data in Action course, we gained the opportunity to explore a crucial question in terms of data and social impact: What does it take to use data responsibly in service of social good? Over the semester, we explored the legal, ethical, and historical dimensions of data use, while partnering directly with local non-profits to co-create a data project that reflected their values, needs, and mission. We learned to critically examine how data is produced and interpreted, and how thoughtful design and communication can make data more useful. Through hands-on work, we gained insight into both the power and the responsibility that comes with using data in the public sphere.
Our team partnered with the Philadelphia Solar Energy Association (PSEA), a non-profit that promotes solar energy adoption across Pennsylvania through advocacy, education, and community engagement. PSEA’s main challenge was related to data collection and visualization. Solar installation data was scattered across different platforms, inconsistently formatted, and difficult to update. This limited their ability to create timely, effective visual materials to inform the public and support clean energy initiatives. The goal with our project was to streamline the data collection and visualization process that was often compiled by one person. We developed a sustainable, code-based process to gather, clean, and visualize solar data from public sources like AEPS, SEIA, and PJM. Using Python, we created scripts that automated data extraction and analysis, providing a final deliverable of user-friendly, updatable plots delivered in a Jupyter Notebook format. Each team member contributed to the project in a unique way. One member focused on scraping and organizing the data, experimenting with different Python libraries to handle inconsistent formats and shifting web structures. Another led the visualization efforts, creating clear and interpretable charts like histograms, bar graphs, and bubble plots to illustrate trends in solar adoption. Other teammates documented the full workflow and assembled the project deliverables, ensuring our work would be easy for PSEA to maintain long-term. Throughout the semester, we met biweekly with PSEA staff to present our progress and adapt our approach based on their needs. By the end, we had a working system that helped streamline their outreach efforts and gave us a real sense of what it means to do data work that matters.
One experience that stood out during this project was the pivot in our final project deliverable format. In the beginning, our team members aimed to create a product that would require no work on the back-end from PSEA. This manifested in the use of an API that would run visualizations based on our Python scripts and deploy them to a separate website. In discussing with our supervisors, we decided that this format would ultimately not serve PSEA’s goals, so we pivoted to Jupyter Notebook. Initially, this felt like a setback because the scripts then required some efforts from PSEA to download external data sources. However, in troubleshooting this issue, we wrote documentation for the data import process. Our final deliverable decreases the overall workload for PSEA, if not being 100% hands-off, and this experience gave us deeper insight into how crucial it is to openly communicate with partners about technical limitations and updates.
This course and partnership with PSEA allows us to gain technical skills as well as tools for data analysis, collaboration, and project design. We learned how data can shine a light on possibilities for community advocacy, and we’ll take with us the ability to communicate our work clearly while handling data responsibly.