Detailed Program

We are witnessing a remarkable comeback of computer programming in schools. While computers seem to be accessible everywhere, particularly outside school, where children and youth are connecting to wider networks of other young users, their capacity to wield such devices critically, creatively, and selectively is decidedly less potent. Learning the language of computers introduces students to processes for thinking and solving problems and for engaging creatively and critically with digital media. Using the example of electronic textiles—computational artifacts in which circuits are stitched with conductive thread to connect microcontrollers and control sensors and actuators, I describe how we can broaden access, diversify representation, and deepen learning. More importantly, I illustrate how the introduction of crafting with computing in schools that serve marginalized populations can help diversify who makes and the kinds of artifacts that are being made, how projects can introduce more complex programming concepts and practices, and what kind of tools we can design to support novice programmers in coding and debugging hybrid artifacts.

About the Speaker

Yasmin Kafai is a professor at the University of Pennsylvania. She is a learning designer and researcher of online tools, projects and communities to promote computational making, crafting, and creativity. With colleagues at MIT, she developed the programming language Scratch and researched applications and participation in clubs, classrooms, and online communities. With Jane Margolis and Joanne Goode, she developed electronic textile projects to introduce computing, crafting, and engineering to high school students and teachers as part of the nationwide “Exploring Computer Science” curriculum. She has written several books, among them “Connected Code: Why Children Need to Learn Programming,” “Connected Gaming: What Making Videogames Can Teach Us About Learning and Literacy,” and “Connected Play: Tweens in a Virtual World”—all published by MIT Press. Kafai earned a doctorate from Harvard University while working with Seymour Papert at the MIT Media Lab. She is a Fellow of the American Educational Research Association and the International Society for the Learning Sciences.

Almost all software development revolves around the discovery and use of application programming interfaces (APIs). Once a suitable API is selected, programmers must begin the process of determining what functionality in the API is relevant to a programmer’s task and how to use it. Our work aims to understand how API functionality is discovered by programmers and where tooling may be appropriate. We employed a mixed-methods approach to investigate Apache Beam, a distributed data processing API, by mining Beam client code and running a lab study to see how people discover Beam’s available functionality. We found that programmers’ prior experience with similar APIs significantly impacted their ability to find relevant features in an API and attempting to form a top-down mental model of an API resulted in less discovery of features.

Manual image labeling (selecting an appropriate “category” for an image) is very tedious and time consuming especially when selecting labels from a large number of categories. In this study, we propose a hierarchical assignment of labeling tasks where the labelers recursively classify images in a category group into sub category groups, working on a single level at a time. This significantly makes each labeler’s task easier, reducing the number of choices from 1,000 to 27 on average. In the user study, we compared our hierarchical assignment to a normal (non-hierarchical) assignment for a labeling task. The results show that the hierarchical assignment requires less total time to complete the labeling task. In addition, the learning effect in the labeling process is more profound in the hierarchical assignment.

The need for learning about emerging technologies and technical concepts among the general population is increasing, but formal explanations from online encyclopedias, textbooks, and articles are often rich in technical jargon and can be challenging to grasp. In this paper, we explore a novel approach for explaining technical concepts to non-technical users through the design of JargonLite, an interactive dictionary that shows how technical concepts can be used in everyday conversations. We evaluated curated conversation-driven explanations with 12 users who had little or no technical training and wanted to learn about big data concepts. We found that users perceived conversation-driven explanations to be easy to understand and these explanations helped them to maintain focus. We discuss the potential of using conversation-driven explanations as a learning tool for non-technical users and reflect on the advantages and disadvantages of this approach.

We present a study that characterizes the way developers use automatically generated patches when fixing software defects. Our study tasked two groups of developers with repairing defects in C programs. Both groups were provided with the defective line of code. One was also provided with five automatically generated and validated patches, all of which modified the defective line of code, and one of which was correct. Contrary to our initial expectations, the group with access to the generated patches did not produce more correct patches and did not produce patches in less time. We characterize the main behaviors observed in experimental subjects: a focus on understanding the defect and the relationship of the patches to the original source code. Based on this characterization, we highlight various potentially productive directions for future developer-centric automatic patch generation systems.

Scientists about their programming practices, and the extent to which they adhere to six common best practices. We argue that these practices are essential to the core scientific value of reproducibility. Our results indicate that many of these practices are not followed because of barriers such as low self-efficacy and misaligned incentive structures. We conclude with suggested improvements to the tooling, education, and incentives of scientific programmers.

This study investigated the essential of meaningful automated feedback for programming assignments. Three different types of feedback were tested, including (a) What’s wrong- what test cases were testing and which failed, (b) Gap comparisons between expected and actual outputs, and (c) Hint hints on how to fix problems if test cases failed. 46 students taking a CS2 participated in this study. They were divided into three groups, and the feedback configurations for each group were different: (1) Group One -What’s wrong, (2) Group Two -What’s wrong + Gap, (3) Group Three -What’s wrong+ Gap + Hint. This study found that simply knowing what failed did not help students sufficiently, and might stimulate system gaming behavior. Hints were not found to be impactful on student performance or their usage of automated feedback. Based on the findings, this study provides practical guidance on the design of automated feedback.

Systems understanding is a skill required to solve many of the world’s most important problems, from climate change to immunotherapy to social decision-making. However, these problems also require communication among experts with diverse skill sets and academic backgrounds. Our long-term goal is to facilitate systems understanding across a range of disciplines through end-user computational modeling tools. This paper presents the Ceptre Editor, a structure editor for the rule-based programming language Ceptre. The Ceptre Editor runs in the browser and offers a visual interface and integrated development environment for Ceptre, following design recommendations from end-user programming, with the goal of providing discoverable affordances for program construction and maintaining syntactic well-formedness at each edit state. We performed a preliminary evaluation of the tool through a qualitative study, assessing the editors effectiveness at helping users understand and extended a system model, and found promising results regarding learnability and mental model accuracy.

The growing popularity of machine learning (ML) has attracted more software developers to now want to adopt ML into their own practices, through tinkering with and learning from ML framework websites and online code examples. To investigate the motivations, hurdles, and desires of these software developers, we deployed a survey to the website of the TensorFlow.js ML framework. We found via 645 responses that many wanted to learn ML for aspirational reasons rather than for immediate job needs. Critically, developers faced hurdles due to a perceived lack of mathematical and theoretical background. They desired frameworks to provide more basic ML conceptual support, such as a curated corpus of best practices, conceptual tutorials, and a de-mystification of mathematical jargon into practical tips. These findings inform the design of ML frameworks and informal learning resources to broaden the base of people acquiring this increasingly important skill set.

Online CS education shows many gender-inclusivity problems in practice and through tools, and little has been done to address this. In CS classrooms, pair programming has been shown to significantly help women and men understand and appreciate programming concepts, and to help close gender gaps. Unfortunately, pair programming is not well supported in online CS education, especially when one of the pair members is a woman. Our overall objective is to address this gap by investigating the following question: How can we bring the educational benefits of pair programming to online CS students in gender-equitable ways? In this paper, we empirically investigate whether and how technology-mediated remote pair programming hinders online students of same- and mixed-gender pairs. Based on our results, we propose refining personas and the cognitive walkthrough to include leadership styles and preferences for pair-programming roles. We further recommend features for online CS educational tools to promote gender-inclusiveness.

Code-first approaches for introducing students to CS exclude those without preparatory privilege in programming and those intimidated by coding. Delaying coding or not using coding in an introductory CS course provides an equitable learning opportunity and includes a broader group of students in computational education. We present a study that compares a traditional Python code-first approach with an approach to delay or remove coding by first using simple, well-known stories to explain computation without the need for a computer or coding. We find that many students, especially female students and those without prior programming, are initially not interested in coding but in using stories to explain computing. We conclude that a traditional Python code-first approach excludes these students and an option using stories is a viable alternative.

Text languages are perceived by many computer science students as difficult, intimidating, and/or tedious in nature. Conversely, blocks-based environments are perceived as approachable, but many students see them as inauthentic. Bidirectional hybrid environments provide textual and blocks-based representations of the same code, thereby offering students the opportunity to seamlessly transition between representations to build a conceptual bridge between blocks and text. However, it is not known how use of hybrid environments impacts perceptions of programming. To investigate, we conducted a study in a public middle school with six classes (n=129). We found that students who used hybrid environments perceived text more positively than those who moved directly from blocks to text. The results of this research suggest that hybrid programming environments can help to transition students from blocks to text-based programming while minimizing negative perceptions of programming.

Efforts to improve diversity in computing have mostly focused on K-12 and university student populations, so there is a lack of research on how to provide these benefits to adults who are not in school. To address this knowledge gap, we present a case study of how a nine-member team of end-user programmers designed an educational program to bring job-relevant computing skills to adult populations that have traditionally not been reached by existing efforts. This team conceived, implemented, and delivered Cloud Based Data Science (CBDS), a data science course designed for adults in their local community in historically marginalized groups that are underrepresented in computing fields. Notably, nobody on the course development team was a full-time educator or software engineer. To reduce the amount of time and cost required to launch their program, they repurposed end-user programming skills and tools from their professions, such as data-analytic programming and reproducible scientific research workflows. This case study demonstrates how the spirit of end-user programming can be a vehicle to drive social change through grassroots efforts.

Technical interviews—a problem-solving form of interview in which candidates write code—are commonplace in the software industry, and are used by several well-known companies including Facebook, Google, and Microsoft. These interviews are intended to objectively assess candidates and determine fit within the company. But what do developers say about them?To understand developer perceptions about technical interviews, we conducted a qualitative study using the online social news website, Hacker News—a venue for software practitioners. Hacker News posters report several concerns and negative perceptions about interviews, including their lack of real-world relevance, bias towards younger developers, and demanding time commitment. Posters report that these interviews cause unnecessary anxiety and frustration, requiring them to learn arbitrary, implicit, and obscure norms. The findings from our study inform inclusive hiring guidelines for technical interviews, such as collaborative problem-solving sessions.

In live-streamed programming, developers broadcast their development work on open source projects using streaming media such as YouTube or Twitch. Sessions are first announced by a developer acting as the streamer, inviting other developers to join and interact as watchers using chat. To better understand the characteristics, motivations, and challenges in live-streamed programming, we analyzed 20 hours of live-streamed programming videos and surveyed 7 streamers about their experiences. The results reveal that live-streamed programming shares some of the characteristics and benefits of pair programming, but differs in the nature of the relationship between the streamer and watchers. We also found that streamers are motivated by knowledge sharing, socializing, and building an online identity, but face challenges with tool limitations and maintaining engagement with watchers. We discuss the implications of these findings, identify limitations with current tools, and propose design recommendations for new forms of tools to better supporting live-streamed programming.

This paper presents the design of Blocks4DS, a block-based environment for students to learn data structures. As a proof-of-concept, we designed custom blocks to allow students to build and visualize Binary Search Trees (BST). Blocks4DS is built on Blockly and uses vis.js to provide visualizations of the binary search tree and its operations. This paper describes the results from an initial evaluation of usability and student learning.

Good documentation has long been argued to be key to helping developers write code more quickly and consistently with design decisions, but is left largely disconnected from code. We propose a method for active documentation, where design decisions are made explicit as design rules and checked against code. Developers can discover how to follow a design rule by navigating to examples in their codebase. After editing code, developers receive immediate feedback about which design rules are satisfied and which are violated, notifying developers who miss design decisions about the existence of these design decisions. We implemented our approach in a prototype tool and conducted a user study. Compared to developers using a traditional design document, developers working in an unfamiliar codebase with active documentation were faster and more successful, using active documentation to learn how to follow design decisions through examples and receive immediate feedback on their changes.

In today’s world, learning new APIs (Application Programming Interfaces) is fundamental to being a programmer. Prior research suggests that programmers learn on-the-fly while they work on other project-related tasks. Yet, this process is often inefficient. This inefficiency has inspired research seeking to understand and improve API learnability. While the existing research has provided insight into API learning, we still have a fractured understanding of the process of learning a new API. In this paper, we take the first steps towards developing a theoretical model of API learning by combining predictions from Information Foraging Theory (IFT) to describe information search behavior, Cognitive Load Theory (CLT) to describe learning, and External Memory (EM) to describe how API learners augment their short term memories. Our proposed model is consistent with existing research on barriers to learning APIs and helps to provide explanations for these barriers as well as suggest new research directions.

Spreadsheets are frequently used in industry to support critical business decisions. Unfortunately, they also suffer from error-proneness, which sometimes results in costly consequences. Experiments in the field of program education have shown that programmers tend to make fewer errors and can better focus on the logic of a program if they use a block-based language instead of a textual one. We hypothesize that a block-based formula editor could support spreadsheet users in a similar way. Therefore, we develop XLBlocks and conduct a think-aloud study with 13 experienced spreadsheet users from industry. Participants are asked to create and edit several formulas, using our block-based language. We then ask them to evaluate this editor using the Cognitive Dimensions of Notations framework. We found that for all dimensions the block-based formula editor received a better evaluation than the default text-based formula editor.

How can software practitioners assess whether their software supports diverse users? Although there are empirical processes that can be used to find “inclusivity bugs” piecemeal, what is often needed is a systematic inspection method to assess software’s support for diverse populations. To help fill this gap, this paper introduces InclusiveMag, a generalization of GenderMag that can be used to generate systematic inclusiveness methods for a particular dimension of diversity. We then present a multicase study covering eight diversity dimensions, of eight teams’ experiences applying InclusiveMag to eight under-served populations and their “mainstream” counterparts.

Modern approaches to introductory Computer Science education utilize block-based programming languages heavily, which poses a problem, as students latter have to switch to textual programming languages for real-world programming. We present the Poliglot environment that aims to smooth this transition by presenting the program simultaneously in both notations, and allows editing in either one. The environment features an educational programming language, Limpid, which we developed for this project. We evaluated it using a preliminary user study on primary school students. The responses were encouraging and we also noted a spontaneous shift from block to text-based programming.

Block-based programming has been overwhelmingly successful in revitalizing introductory computing education and in facilitating end-user development. However, poor code quality makes block-based programs hard to understand, modify, and reuse, thus hurting the educational and productivity effectiveness of blocks. There is great potential benefit in empowering programmers in this domain to systematically improve the code quality of their projects. Refactoring–-improving code quality while preserving its semantics–-has been widely adopted in traditional software development. In this work, we introduce refactoring to Scratch. We define four new Scratch refactorings: Extract Custom Block, Extract Parent Sprite, Extract Constant, and Reduce Variable Scope. To automate the application of these refactorings, we enhance the Scratch programming environment with powerful program analysis and transformation routines. To evaluate the utility of these refactorings, we apply them to remove the code smells detected in a representative dataset of 448 Scratch projects. We also conduct a between-subjects user study with 24 participants to assess how our refactoring tools impact programmers. Our results show that refactoring improves the subjects’ code quality metrics, while our refactoring tools help motivate programmers to improve code quality.

Computer science has become an increasingly visual discipline. Tools in a wide variety of domains, from gaming environments to programming or scientific ones, use graphics for a variety of purposes (e.g., entertainment, knowledge acquisition, general user interactions). While the impact is clear enough for the general population, the disabilities community can have significant challenges in using such content, especially the blind or visually impaired. In this talk, we will discuss the research challenges involved in making graphical and visualization technologies accessible. This will include discussion of new techniques for designing interactive graphical applications that are usable even by those that cannot see the screen.

About the Speaker

Andreas Stefik is an associate professor in computer science at the University of Nevada, Las Vegas. For the last decade, he has been creating technologies that make it easier for people, including those with disabilities, to write computer software. He helped establish the first national educational infrastructure for blind or visually impaired students to learn computer science and invented the first evidence-oriented programming language, Quorum. The design of Quorum is based on data derived through methodologies similar to those used in the medical community. He has been a principal investigator on 5 NSF-funded grants, many of which related to accessible graphics and computer science education. Finally, he was honored with the 2016 White House Champions of Change award and the Expanding CS Opportunities award from Code.org and the Computer Science Teachers Association.

Though support for learning computing in schools is growing, many children still begin learning to program without formal support in open-ended programming environments. While researchers have evaluated the final code of these types of projects, we know little about how users’ behaviors and usage of support tools relate to understanding. We ran a study where participants had open-ended programming time with access to one of two support tools: suggestions or tutorials. Participants then completed four tasks which required understanding of the suggestion or tutorial content. We did not find an effect of suggestions compared to tutorials on knowledge application, but we did find that many participants who performed better tended to explore more of the interface, code behaviors, and support tools. Our results suggest that future tools for encouraging learning during open-ended programming should likely focus on supporting users who tend to explore less on their own.

Integrated development environments (IDEs) are ubiquitous in software development. Despite their popularity, much of their designs are not based on empirical findings and have been virtually unchanged since their inception. More recently, researchers have proposed alternative IDE designs based on observational studies of developers and found promising benefits. However, many design decisions are still unsupported by empirical evidence. Towards designing an empirically-driven IDE, we performed an analysis on code size and screen space. First, we analyzed the size of code from 95 projects in 4 programming languages. Second, we calculated how much code can fit onscreen using various document arrangements in VS Code with common screen resolutions. We found (1) the length of code depends on the programming language, while the width of code is similar across languages and (2) the amount of code onscreen can be substantially increased by properly configuring the IDE and by using a high-resolution monitor.

Creating Virtual Reality (VR) applications normally requires advanced knowledge of imperative programming, 3D modeling, reactive programming, and geometry. Immersive authoring tools propose to reduce the learning curve of VR programming by allowing users to create VR content while immersed in VR. Immersive authoring can take advantage of many of the features that make VR applications intuitive and natural to use—users can manipulate programming primitives through direct manipulation, immediately see the output of their code, and use their innate spatial reasoning capabilities when viewing a program. In this paper, we investigate the benefits and challenges of immersive dataflow authoring. We implemented an immersive authoring tool that enables dataflow programming in VR and conducted a series of retrospective interviews. We also describe design implications for future immersive authoring tools.

Females, racial minorities, and other diverse individuals face many challenges in learning to program. Researchers have investigated these challenges for many years, yet participation by these groups continues to be low. Our action research project is investigating a non-formal learning activity for female and gender-diverse faculty, staff, and students in a university setting. Using surveys, observations and focus groups, we probed learners’ experiences, to examine the short- and longer-term impacts of the workshop. We observed increases in self-efficacy and predictions of future use of programming; trepidation about computing decreased. We discuss the need to offer learning spaces for skills that can empower members of these groups.

Programmers are expected to use multiple programming languages frequently. Studies have found that programmers try to reuse existing knowledge from their previous languages. However, this strategy can result in misconceptions from previous languages. Current learning resources that support the strategy are limited because there is no systematic way to produce or validate the material. We designed three instruments that can help identify and validate meaningful behavior differences between two languages to pinpoint potential misconceptions. To validate the instruments, we examined how Python programmers predict behavior in a less familiar language like R, and whether they expect various R semantics. We found that the instruments are effective in validating differences between Python and R which were linked to misconceptions. We discuss design trade-offs between the three instruments and provide guidelines for researchers and educators in systematically validating programming misconceptions when switching to a new language.

Developers authoring HTML documents define elements following patterns which establish and reflect the visual structure of a document, such as making all images in a footer the same height by applying a class to each. To surface these patterns to developers and support developers in authoring consistent with these patterns, we propose a mixed human-AI technique for creating code patterns. Patterns are first learned from individual HTML documents through a decision tree, generating a representation which developers may view and edit. Code patterns are used to offer developers autocomplete suggestions, list examples, and flag violations. To evaluate our technique, we conducted a user study in which 24 participants wrote, edited, and corrected HTML documents. We found that our technique enabled developers to edit and correct documents more quickly and create, edit, and correct documents more successfully.

One of the greatest achievements of Scratch as an educational tool is the eager willingness of programmers to use existing projects as the starting point for their own projects, a practice known as remixing. Despite the importance of remixing as a foundation of collaborative and communal learning, the practice remains poorly understood, with the Scratch programming community remaining in the dark about which programming practices encourage and facilitate remixing. Scratch designers lack feedback on how the remixing facility is used in the wild. To gain a deeper insight into remixing, this paper investigates heretofore unexplored dimensions of remixing: (1) the prevailing modifications that remixes perform on existing projects, (2) the impact of the original project’s code quality on the granularity, extent, and development time of the modifications in the remixes, and (3) the propensity of the dominant programming practices in the original project to remain so in the remixes. Our findings can encourage remixing and improve its effectiveness, benefiting the educational and end-user programming communities.

With the increasing need to teach programming to novices using collaborative methods like pair programming, it is important to understand how different input devices can help support collaborative learning. In this paper we present Multi-Device Grace, the first application to explore block-based programming in a cross-device environment consisting of digital tabletops, mobile tablets, and laptops. We conducted a user study (n=18) to explore how cross-device environments can support co-located collaborative block-based programming. The study used Tiled Grace, an existing block-based programming language, and our extensions: Tabletop Grace (designed for tabletops) and Mobile Grace (designed for tablets). Our results show that the majority of participants felt they were able to collaborate quickly and easily, and the cross device interaction would be particularly beneficial in an education setting.

Many errors in spreadsheet formulas can be avoided if spreadsheets are built automatically from higher-level models that can encode and enforce consistency constraints. However, designing such models is time consuming and requires expertise beyond the knowledge to work with spreadsheets. Legacy spreadsheets pose a particular challenge to the approach of controlling spreadsheet evolution through higher-level models, because the need for a model might be overshadowed by two problems: (A) The benefit of creating a spreadsheet is lacking since the legacy spreadsheet already exists, and (B) existing data must be transferred into the new model-generated spreadsheet. To address these problems and to support the model-driven spreadsheet engineering approach, we have developed a tool that can automatically infer ClassSheet models from spreadsheets. To this end, we have adapted a method to infer entity/relationship models from relational database to the spreadsheets/ClassSheets realm. We have implemented our techniques in the HaExcel framework and integrated it with the ViTSL/Gencel spreadsheet generator, which allows the automatic generation of refactored spreadsheets from the inferred ClassSheet model. The resulting spreadsheet guides further changes and provably safeguards the spreadsheet against a large class of formula errors. The developed tool is a significant contribution to spreadsheet (reverse) engineering, because it fills an important gap and allows a promising design method (ClassSheets) to be applied to a huge collection of legacy spreadsheets with minimal effort.