Hacking the Harvard Kennedy School Core — By Getting Out Of The Classroom

Class Photo (Spring 2023)

This spring was my seventh time teaching at Harvard Kennedy School. Yet in many ways, I felt like a new instructor — experimenting, failing, and learning alongside almost 60 Master in Public Policy (MPP) students.

It was my second year teaching a relatively new class on policy implementation (API-502B), and I wanted to introduce the experiences and clients from my past field elective. As context, for five years before the pandemic, I taught DPI-663: Tech and Innovation in Government, a field class that paired student teams with government clients to solve real-world problems. The student teams conducted field research, prototyped and tested possible solutions, and presented their findings to their clients at the end of the semester — learning teamwork, human-centered design, product management, storytelling, and public sector entrepreneurship. (Check out the DPI-663 class website.)

This spring’s class, Policy Implementation (API-502B), on the other hand, was a required class — known as “core class” — for all first-year Masters in Public Policy (MPP) students to learn about the implementation of government policy. Eric Rosenbach, Jack Donahue, Afreen Siddiqui, and I designed the initial curriculum — covering topics from systems thinking to project management to financial management — and taught it initially in spring 2022. The first time we taught the class it didn’t feel like a roaring success, partly because we tried to cover too many topics — though in fairness every new core class is a bit rocky in its first year.

This spring (2023), each instructor was given a bit more latitude, and so I focused the class on government service delivery and issues of digital government. Topics included user-centered design, prototyping, product management, ethics, risk management, budgeting, metrics, communications, and more. But I believe that students learn more through hands-on experiential learning, especially in teams, because they can learn from each other and their clients. So we also partnered with clients from the City of Boston, the Consumer Financial Protection Bureau (CFPB), the U.S. Census Bureau, the U.S. Department of Veteran Affairs (VA), the Centers for Medicare & Medicaid Services (CMS), and the German Ministry of Labour and Social Affairs.

Despite the challenges of trying to incorporate some elements of a field class at scale, each student team found creative ways to research human needs, build and test prototypes with real people, and understand the complexity and constraints of government execution. They then delivered a strong presentation and implementation plan for their client. I’m hopeful each student came away with a learning experience that they carry with them throughout their careers.

I had many distinguished guests in class (see further below in this post for a longer list), but my favorites were my ten-year-old twin daughters, Ellis and Georgia.

Above and Below: My 10-year-old daughters’ class notes (also here and here)

Georgia drawing on my office dry-erase board

CLASS PROJECTS
Here is a summary of what the students worked on with their government clients:

City of Boston — Tax Exemptions:

Three student teams — Sophia Boyer, Anahita Sahu, and Er Li Peng on the first team, Jalen Benson, Michael Reschke, and Jennifer Pfister on the second team, and Angela Wu, James Calello, and Juan Mejia on the third team — partnered with the City of Boston to improve awareness and implementation of the City of Boston’s homeowners’ tax exemption, especially among low-income households to improve housing affordability. The teams analyzed publicly available data and conducted over 44 interviews with Boston property owners, real estate companies, homeowner associations, and community groups. The students discovered that many city residents are unaware of the tax exemption and that higher-income homeowners are more likely to take advantage of the exemption. They learned the exemption process is difficult to navigate and not well-integrated with either home buying or tax filing. The teams took different approaches (see here, here, and here) but all designed, tested, and developed two core ideas:

• Simpler digital forms to improve the current process of applying for homeownership tax exemption; and
• A campaign to raise awareness of the homeowner tax exemption among Boston city residents.

Jennifer Pfister conducts use research with residents of Boston from the Harvard Kennedy School

James Calello talking with a real estate agent to understand the tax exemption process

Er Li Peng presents her team’s research findings and plan for the City of Boston, by starting with the story of a long-time resident

City of Boston — Street Sweeping:

One student team — Tasila Banda, Huw Spencer, and Emily Tench — partnered with the City of Boston to reduce the number of cars towed due to street sweeping in the City of Boston. The team designed a new street sweeping and towing alert feature to integrate into the ParkBoston app as well as a plan for the City of Boston to pilot, scale up, and integrate the program into normal operations.

Team deliverable: Mapping how Boston residents experience the sweeping and towing process

Consumer Financial Protection Bureau (CFPB) — Connecting citizens with CFPB services:

One student team — Caroline Elmendorf, Sandhya Jetty, and Tom McGee — explored the idea of better support for victims of predatory practices. The team conducted over 18 interviews in the Boston area, asking respondents about their experience with predatory practices, their familiarity with government resources, and their knowledge of the CFPB. The team discovered that most people don’t understand how the CFPB or other agencies might help. The team prototyped 1) a simple tool to direct users to the government agency best placed to handle a complaint and 2) an interactive web assistant for the CFPB website — called George the Eagle — to guide users to the right source of government help. Their plan for CFPB is here.

User interviews: Sandhya Jetty interviews Boston area residents at MBTA stations

Consumer Financial Protection Bureau — Complaint Awareness:

One student team — Rehan Adamjee, Kabir Mehta, David Merkel, and Christos Porios — looked at helping consumers address credit report inaccuracies in a more pragmatic and timely manner. The team analyzed publicly available data on complaints and conducted 11 interviews to understand user pain points in registering complaints. The team created a plan for an awareness campaign for how to access CFPB’s services and file complaints, a Google Ad campaign to better assist potential victims, and better triage of complaints.

Team deliverable: Newly designed awareness campaign

Census Bureau — API V2.0:

One student team — Aliza Amin, Hugh Grant-Chapman, and Daksh Baheti — focused on the Census Bureau’s initiative to refine its application programming interface (API). They interviewed users and heard about frustrations with documentation, complicated variable architectures, and the lack of comprehensive responses. The team developed a plan to upgrade the API call architecture and re-design Census documentation.

Census Bureau — Data Equity:

Another student team — Kaamila Patherya, Logan Herman, and David Dam — focused on enhancing data equity for the Census’s data portal, data.census.gov. Their user research revealed confusing search results pages, difficulties understanding data tables, and general dissatisfaction with the page’s organization. They developed a plan to simplify the landing page, great flexibility for users, and improved legibility. A second student team — Abeera Ahmed, Aditi Kothari, and Lilly Tong — also focused on data equity and proposed a plan to redesign the website layout, simplify language, and improve functionality. A third student team — Lucas Peilert, Richa Chaturvedi, and Ashley Kyalwazi — explored using new generative artificial intelligence technology on data.census.gov, proposing a plan that included a potential partnership with OpenAI.

Team deliverable: low-fidelity paper prototype of data tables on Census website

Census Bureau — Streamlining Data Tools:

One student team — Moiz Agha, Willie Henderson, and John Yatsko — focused on streamlining the existing suite of tools on Census’s website, making them more accessible and discoverable. They developed a plan for a web redesign project that includes a specific tool search bar as well as directories that list tools by their respective categories.

VA CTO Charles Worthington meeting with students working with the VA on PACT Act Implementation

Department of Veterans Affairs — PACT Act:

The U.S. Department of Veterans Affairs is implementing an expansion of the PACT Act, legislation passed in 2022 that stands as one of the most significant advancements in veteran benefits in decades. The student teams working with the VA, several of which included Veterans themselves, learned that many Veterans were not aware of the benefits expansion under the PACT Act and that many were unfamiliar with the eligibility requirements. To address this, the teams proposed alternative methods to increase awareness to improve the number of eligible veterans who apply for PACT Act benefits. One student team — Bryn McLaughlin, Caroline Sabatt, and Michelle Roca — developed a plan for a three-pronged marketing campaign, including email and mail outreach, celebrity engagement through various media outlets, and hosting in-person educational events at Veteran Service Organizations (VSO) and VA Clinics throughout the country. Another student team — Charles Kelly, Ilai Levin, Jake Steckler, and Samriddhi Vij — developed a plan for a prototype screening tool to help veterans determine their eligibility for benefits. A third student team — Jebb Ricketts, Paul Nolan, and Sean Norick Long — conducted research and prototyping to develop a plan for a digital ad campaign designed to increase awareness of benefits for U.S. veterans and their families.

Team deliverable: PACT ACT screening prototype that includes those applying on behalf of family members

Centers for Medicare & Medicaid Services — Improving Medicaid Enrollment:

Five student teams worked with the Centers for Medicare and Medicaid Services (CMS) to design a prototype of a model application for State Medicaid Offices that reduced the administrative burden for eligible applicants. Interestingly, each team approached this challenge in very different ways:

• One team — Noam Alon, Bella Zhou, and Jianing Wu — took to Facebook to interview people across Virginia, Arkansas, New York, New Jersey, and Illinois. Their research indicated that jargon, poor online support, and unclear processes created confusion, particularly for low-income individuals and marginalized groups who were most likely to utilize Medicaid. The team prototyped a new application portal with four key features: (1) simple, clear language, (2) streamlined user interfaces that leverage skip-logic to enable applicant-specific questions, (3) an AI chatbot that provides real-time support, and (4) a notification feature that provides applicants with updates via text and email. Here is their plan for CMS.
• Another team — Naomi Eisenberg, Maya Alper, and Meisui Liu — focused on users who often fall through the cracks: those who are unhoused, justice-impacted, experiencing emergencies, and domestic violence survivors. As a result, their research was focused on navigators — e.g social workers who support Medicare enrollment — who often deal with the most difficult cases in the system and push it to its limits. This team proposed to pilot a standardized online enrollment platform in three states before expanding across the United States. Their prototype provides step-by-step tips to mimic human help, simplifies the application process, and improves the document verification process. Here is their plan for CMS.
• One team — Bryan Fores, Shashwat K, and Keaton Lee — interviewed people from Craigslist and Reddit and found that many respondents find Medicaid applications confusing. To address this, the team developed a plan for reducing the administrative burden for future applications, particularly those from marginalized communities.
• One team — Linor Zeilig and Safiyah Bharwani — recruited participants from seven states using Craigslist, and found that awareness of Medicaid services and eligibility and accessibility in terms of language and technology were the greatest barriers to service uptake and enrollment. As a result, this team developed a plan to answer eligibility concerns via 5 basic questions and devised a multi-channel marketing campaign in English and Spanish.

Linor Zeilig and Safiyah Bharwani, working with CMS, presenting their research

German Ministry of Labour and Social Affairs — Make-it-in-Germany.com:

One student team — Yunus Berndt and Eva Richter — found their own client in the German Ministry of Labour and Social Affairs, which wanted to improve the reach and user experience of make-it-in-germany.com, the Ministry’s primary portal to attract highly skilled foreign labor to Germany. The team interviewed master’s students within the Harvard community, asking them to navigate the internet to figure out how they could work in Germany. The team also performed desk research on similar websites in Sweden and Australia. Their research identified significant issues, including poor search engine optimization, poor translations, defunct pages, and too much information. The team redesigned a new prototype website for the Ministry and developed an affordable implementation plan.

Clients
The clients of API-502B were mostly organizations and people who have worked with my past field class. I appreciate government clients who recognize that the design process may yield answers they don’t anticipate and are open to new ideas and solutions that the student teams generate — even in a class focused on teaching implementation. Some clients — like the City of Boston and the VA — have been collaborating with me since I started teaching in 2016. I’m so grateful to the amazing people in all of our past and current clients who have collaborated with us. It’s especially amazing when students go on to work with our clients in a summer or full-time capacity. Keep hiring my students!

The class is designed to create an intense learning environment for students, first and foremost. But I also aspire for them to build authentic relationships with their clients and generate real insights and impact. Here is what the clients had to say:

• “Good public policy only matters if it is implemented effectively, and the most effective government leaders understand this. It was great to see HKS students dive in and grapple with the details of implementation!” — Charles Worthington, CTO, VA
• “Not only did we gain valuable insights from the student’s user research with Medicaid beneficiaries and outputs, but we ended up having one student, Naomi, spend the summer with us and continue to make valuable contributions to our team and the people we serve.” — Andrea Fletcher, Chief Digital Strategy Officer, Centers for Medicare and Medicaid Services
• “Often in Government, we find ourselves responding to crisis, but we can’t take the time to take a step back to look at problems from a different perspective. The students at HKS did an amazing job of using both qualitative and quantitative research to reframe property tax exemptions and parking and give the City a fresh approach. They presented their recommendations with clear objectives and evaluation metrics, research, and new perspectives. Their work will help the City explore further things like visitor parking permits and other policy innovations.” — Santi Garces, Chief Information Officer, City of Boston
• “Not only was the thinking and research immediately useful in our work, we ultimately were able to connect our teams with sister agencies to benefit from the insight as well!” — Erie Meyer, Chief Technologist, CFPB

Students learning the KJ Method

Students in API-502B

Students came from different fields, including government, military, non-profits, consulting, tech, design, public health, medicine, and more. The class was composed of students who hailed from countries across the globe, such as India, Pakistan, China, Greece, the United Kingdom, Germany, Israel, Venezuela, Canada, Australia, and the United States. As a core class for the HKS MPP curriculum, API-502B was only offered to first-year MPP students. However, almost a quarter were joint or concurrent degree students with other Harvard or partner institution schools, including Harvard Business School, Harvard Medical School, MIT Sloan School of Management, Stanford Graduate School of Business, and Georgetown University Law School. The diversity of backgrounds, experiences, and perspectives enriched class discussions and made for a more interesting class for the students and teaching team alike.

Students after a class exercise

Students during the Marshmallow challenge

Many students appreciated the class! Here is some student feedback from a survey at the end of the class:

• “Working with a real-world client was a great way to emphasize and reinforce the lessons of the class. Conducting user interviews to better define the problem, and subsequently prototyping based on that feedback, gives me more confidence in trying to find solutions and implement them in the future.”
• “This was a great experience! Continuous improvement and constant user interaction were very helpful.”
• “I really liked the guest speakers and their sharing of experiences (esp the Greek minister — so cool!) and Nick’s lectures + case study discussions (Aadhar, Trace Together…). Many lessons to take away: I was new to the world of tech and the importance of the policy-execution gap and need for user testing and prototyping will stay with me.”
• “The variety of the class was good — going to the Harvard iLab, user testing in the ‘wild’, using prototyping software, working with our client.”
• “The in-class interviews were extremely diverse and interesting to hear the various backgrounds and experiences of the professionals. This class leveraged more interviews than any other class I’ve taken and I learned so much from this aspect. The exposure to various issues, projects, and designs was very insightful.”
• “I enjoyed the client project and the array of talented guest speakers. That combination both enhanced my learning and opened the door to new professional connections.”
• “I think learning the prototyping tools was quite useful. I love design (as a hobby though I am bad at it) and playing around in Figma was really fascinating. I’ll take this experience with me for sure!”

Some students were frustrated by the class, for a variety of reasons. Here is some of that feedback:

• “The guests are great but sometimes would love to hear more from Professor Sinai :)”
• “I think the core concept of learning implementation through practice with a client project makes a lot of sense. But in the timeframe we have, it becomes rushed and not as fruitful as it can be.”
• “I feel like we glazed over risk management and budgeting. I want to ensure students (1) learn about these key concepts, and (2) apply those concepts to their project. You could consider a class model where a weekly client deliverable aligns with the course theme. For example, “This week we’re learning about budgeting. After the readings and lecture, your task is to build a budget for your client project.” That creates mini-milestones, rather than asking us to apply everything in an implementation plan at the end.”

HBS Professor Mitch Weiss leading a mini-case on the City of Seoul’s experimental use of a metaverse

The Case Method

While I was determined to make a core class more applied and field-based, we still had classes twice a week for 75 minutes. Cases, if designed and taught well, can be an effective learning technique in the classroom. Inspired by my friend Mitch Weiss, an excellent HBS Professor, I wanted to ensure that our case discussions had a central tension that drove vigorous debate. Sometimes it worked well, like when the students argued over the morality of implementing mandatory digital identity (Aadhaar) in India or whether the State of California should try agile software development in its Child Welfare system modernization; other times the students had more similar views and I had to work harder to foster disagreement.

One learning innovation we experimented with this semester was the idea of a “longitudinal” case. These cases were taught traditionally during a single class, but then also referenced throughout the semester, depending on the topic of the class. The three longitudinal cases — the VA mobile app, the Singapore TraceTogether Contact Tracing app, and the Commonwealth Massachusetts Paid Family Medical Leave Act implementation — had interesting artifacts and examples across many topics (product, budgeting, risk management, etc.) we discussed during the semester. We made it a bit harder on ourselves because we were also writing these cases during the spring semester, but the protagonists (Charles Worthington, Devyn Paros, Julia Gutierrez, and Jason Bay) were class acts and provided rich material throughout the semester, even on very short notice.

Mary Ann Brody introduces Human Centered Design at the Harvad i-Lab

Adjunct Lecturer in Public Policy Kathy Pham talking about product management in government

Guests:
I had many — perhaps too many — guests during class. But I think it is valuable to bring different experiences and perspectives to the class, and I like to invite the protagonist of each case. I also like to introduce to the students other HKS faculty who teach electives on digital topics. Thank you to all of the amazing guests — especially the MPP alums — we had this semester:

• Mina Hsiang, Administrator of the U.S. Digital Service (USDS)
• Ryan Panchadasaram, Kleiner Perkins, and former U.S. Deputy CTO
• Mary Ann Brody, Head of Member Experience at Devoted Health, former USDS
• Erie Meyer, Chief Technologist at the Consumer Financial Protection Bureau (CFPB)
• Kathy Pham, HKS Adjunct Lecturer in Public Policy, Former Deputy Chief Technology Officer of the Federal Trade Commission (FTC)
• Jen Palhka, Founder and former executive director of Code for America, former U.S. Deputy Chief Technology Officer
• Charles Worthington, HKS Adjunct Lecturer in Public Policy and CTO at the U.S. Department of Veterans Affairs
• Kyriakos Pierrakakis, Minister of Digital Governance in the cabinet of Kyriakos Mitsotakis, the prime minister of Greece (MPP ‘07)
• Jason Bay, Former Senior Director at GovTech Singapore (Government Digital Services)
• Clare Martorana, Federal Chief Information Officer, Office of Management and Budget
• Mike Wilkening, Former Secretary, California Health & Human Services Agency
• Julia Gutierrez, Chief Digital Officer for the City of Boston (MPP ‘17)
• Devyn Paros, Chief Digital Officer at Commonwealth of Massachusetts (MPP ‘17)
• Mitch Weiss, Professor at Harvard Business School, former Chief of Staff at the City of Boston Mayor’s Office
• Ayushi Roy, HKS Adjunct Lecturer in Public Policy, Deputy Director of the New Practice Lab at New America
• Kumar Garg, Managing Director, Schmidt Futures, and former Assistant Director for Learning Innovation at the Office of Science and Technology Policy

Minister Kyriakos Pierrakakis spoke to the class about digital transformation in Greece.

Teaching Team
This class would not have been possible without the tireless contributions of Tarinee Kucchal, Ed Chao, and Ethan Lundgren. As dedicated course assistants, they organized the class, mentored students, researched and co-wrote cases, and so much more. I’m so grateful for their contribution and wise counsel. The class felt the same:

• “Loved the CAs! Really appreciate their time, effort, and hard work in making this class a worthwhile experience.”
• “CAs were always available and able to answer our questions. Extremely helpful and encouraging as we went through various projects and class items.”

Books
While most of the assigned reading on the syllabus were blogs, short articles, and excerpted chapters, we included a few books in the curriculum this year:

• Hack Your Bureaucracy: Get Things Done No Matter What Your Role On Any Team, by Marina Nitze and me. Let the record show I gave the class my book, rather than making them buy it!
• Recoding America: Why Government Is Failing in the Digital Age and How We Can Do Better, by Jen Pahlka’s book. It is a fantastic book exploring the mismatch between policy and government capability.
• Ryan Panchadasaram provided the class copies of two books including “Measure What Matters” and “Speed & Scale: An Action Plan for Solving Our Climate Crisis Now”. Both are excellent books about setting goals using the Objective and Key Result (OKR) framework.

Even Goose (my dog) was assigned Hack Your Bureaucracy

Implications for the Core and PAE
First, I’m gratified that HKS is including a focus on the actual management and execution of government (as opposed to the traditional focus on policy analysis). But I don’t think it is sufficient to frame API-502 just as the teaching of policy implementation, since the execution of (and entrepreneurship in) government can sometimes lead policy development, rather than just follow from it. In the business world, the analog is the discipline of strategy: most business schools require a course in business strategy, but they also teach management, operations, marketing, entrepreneurship, etc. Similarly, a school of government should of course teach policy, but it should also teach management, operations, public-sector entrepreneurship, etc. Strategy and Policy are about what an organization can and ought to do, but the questions of how to do it are just as worthy of a subject in a graduate school like HKS. Execution matters. To quote from my friend Jen Pahlka’s new book, Recoding America: “Implementation can no longer be policy’s poor cousin. It can’t be beneath the attention of our powerful institutions, and it can’t be beneath our attention as a public.”

Second, I think we need to continue to emphasize applied and experiential learning at HKS, where there are just a handful of field classes. My prior field class (DPI-663) isn’t being offered currently, but I hope some version of it, or something similar, will be offered in future years. I’m a proponent of field classes because they can be effective ways for students to learn valuable professional skills like user research, communications, prototyping, project management, and so much more. I also think they help students build a portfolio of projects to showcase to future employers. It also helps them build relationships with clients, and in some cases, even get hired.

Finally, there is another strong reason to include applied learning in the core: it prepares students for their capstone thesis in their second year, known at the Kennedy School as the Policy Analysis Exercise (PAE). For PAEs, students work with a client to research a policy area or problem and propose a solution. Getting practice working in teams with a client in their first year will set students up for greater success with their PAE.

Conclusion
It is an honor to teach at Harvard Kennedy School, and I’m thankful for the opportunity. I can’t wait to see what these talented HKS students do next in their careers. Check out the students’ great work and please hire them!