这是用户在 2024-7-29 11:32 为 https://ieeexplore.ieee.org/document/8802102 保存的双语快照页面,由 沉浸式翻译 提供双语支持。了解如何保存?
关于计算机科学和编程在线学习偏好的调查 | IEEE 会议出版物 | IEEE Xplore --- A Survey on Online Learning Preferences for Computer Science and Programming | IEEE Conference Publication | IEEE Xplore

A Survey on Online Learning Preferences for Computer Science and Programming
关于计算机科学和编程的在线学习偏好调查

Publisher: IEEE 出版商: IEEE
Javier Escobar-Avila; Deborah Venuti; Massimiliano Di Penta; Sonia Haiduc
哈维尔·埃斯科巴尔-阿维拉;黛博拉·维努蒂;马西米利亚诺·迪·彭塔;索尼娅·海杜克

Abstract: 摘要:

The increasing availability of online documentation is radically changing the way developers and computer science students learn various topics, and programming in partic...
在线文档的日益普及正在从根本上改变开发人员和计算机科学学生学习各种主题的方式,尤其是编程。
View more 查看更多

Abstract: 摘要:

The increasing availability of online documentation is radically changing the way developers and computer science students learn various topics, and programming in particular. Nowadays, a plethora of online learning resources are available, ranging from online University courses and official programming manuals/documentation to video tutorials, screencasts, and discussions available on Question & Answer forums. With the aim of aiding researchers to better understand the usage, advantages, and limitations of this large amount of resources, this paper empirically investigates the learning preferences and needs of computer science students and professional developers related to the use of online resources for learning. We conducted a survey with 205 participants, having different degrees of formal instruction and professional experience in programming. The survey investigated the participants' learning preferences, information seeking behavior, preferred online resources, and learning needs. The study results reveal a preference for combined visual and auditory learning while indicating written tutorials and official documentation as two of the most popular resources currently used by developers and students. The study also indicates that, although online resources are considered valuable, there is a need for better support for searching and browsing this kind of resources.
在线文档的日益普及正在从根本上改变开发人员和计算机科学学生学习各种主题,尤其是编程的方式。如今,有大量的在线学习资源可供使用,从在线大学课程和官方编程手册/文档到视频教程、屏幕录像和问答论坛上的讨论。为了帮助研究人员更好地理解这些大量资源的使用、优势和局限性,本文实证研究了计算机科学学生和专业开发人员在使用在线资源学习方面的学习偏好和需求。我们对 205 名参与者进行了调查,他们在编程方面有不同程度的正规教育和专业经验。调查研究了参与者的学习偏好、信息搜索行为、首选的在线资源和学习需求。 研究结果显示,视觉和听觉相结合的学习方式更受欢迎,同时表明书面教程和官方文档是目前开发者和学生使用最多的两种资源。研究还表明,尽管在线资源被认为是有价值的,但在搜索和浏览此类资源方面需要更好的支持。
Published in: 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET)
发表于:2019 IEEE/ACM 第 41 届国际软件工程会议:软件工程教育与培训 (ICSE-SEET)
Date of Conference: 25-31 May 2019
会议日期:2019 年 5 月 25 日至 31 日
Date Added to IEEE Xplore: 15 August 2019
添加到 IEEE Xplore 的日期:2019 年 8 月 15 日
ISBN Information: ISBN 信息:
Publisher: IEEE 出版商: IEEE
Conference Location: Montreal, QC, Canada
会议地点:加拿大魁北克省蒙特利尔

SECTION I.  第一部分。

Introduction  介绍

The advent of the Internet and, later on, the availability of high-speed connections by many has radically changed the way people learn new concepts [3] . The use of traditional classroom-based teaching along with books and other printed references is nowadays complemented, if not in many cases completely replaced, by distance learning, for example through Massive Open Online Courses (MOOC) [8] and, above all, by the availability of spontaneously-produced online material retrieved through search engines [38] , [71] .
互联网的出现以及后来许多人可以使用高速连接,彻底改变了人们学习新概念的方式 [3] 。传统的课堂教学以及书籍和其他印刷参考资料的使用,如今得到了补充,如果不是在许多情况下完全被远程学习所取代,例如通过大规模开放在线课程(MOOC) [8] ,最重要的是,通过搜索引擎检索到的自发生成的在线材料的可用性 [38] [71]

When it comes to software development, the increasing availability of online resources has changed not only the way students learn computer science (CS) concepts, but also how developers write and document software and how they help others wanting to use it. In fact, studies indicate that nowadays developers spend between 20–30% of their time looking for and acquiring information and code on the Web [5] , [36] , [75] , which is more time than they actually spend coding [6] .
当谈到软件开发时,在线资源的日益普及不仅改变了学生学习计算机科学(CS)概念的方式,也改变了开发人员编写和记录软件以及帮助其他人使用软件的方式。事实上,研究表明,如今开发人员花费 20-30%的时间在网上查找和获取信息和代码 [5] [36] [75] ,这比他们实际编写代码的时间还要多 [6]

A widely adopted online resource is represented by informal documentation on Question & Answer forums, and in particular Stack Overflow (SO). SO provides a large variety of concrete code examples about how to use (or not to use) a certain programming paradigm or piece of technology [54] , [55] , [66] . SO also provides developer insights about the usage of an API [68] and opinions about the suitability of a certain solution or piece of technology [72] , such as whether an API is more reliable or has better performance than another.
一个被广泛采用的在线资源是问答论坛上的非正式文档,特别是 Stack Overflow (SO)。SO 提供了大量关于如何使用(或不使用)某种编程范式或技术的具体代码示例 [54] [55] [66] 。SO 还提供了关于 API 使用的开发者见解 [68] ,以及关于某个解决方案或技术的适用性的意见 [72] ,例如某个 API 是否比另一个更可靠或性能更好。

Online video tutorials are also a popular resource with developers. Software development video tutorials often include screencasts, showing a step-by-step guide of how programming solutions can be implemented — e.g., by displaying code snippets in the Integrated Development Environments (IDEs) — and what the final result of an application is. Previous research by MacLeod et al. [37] has investigated why developers share and use such screencasts, indicating that videos are a useful medium for communicating programming knowledge between developers, and that they present key advantages over written documentation.
在线视频教程也是开发人员的热门资源。软件开发视频教程通常包括屏幕录制,展示如何实现编程解决方案的分步指南——例如,通过在集成开发环境(IDE)中显示代码片段——以及应用程序的最终结果。MacLeod 等人之前的研究 [37] 调查了开发人员为什么分享和使用这些屏幕录制,表明视频是开发人员之间传递编程知识的有用媒介,并且它们相对于书面文档具有关键优势。

Stemming from this rapidly-changing landscape in software development learning, our study intends to investigate learning habits in CS and more specifically for programming. We performed a survey that involved 205 participants, including university students as well as experienced practitioners. More specifically, the survey investigates:
源于软件开发学习中这种快速变化的环境,我们的研究旨在调查计算机科学中的学习习惯,特别是编程方面的学习习惯。我们进行了一项调查,涉及 205 名参与者,包括大学生和有经验的从业者。更具体地说,调查研究了:

  1. the format of the resources participants prefer to use when learning a new topic, categorized using the dimensions described in Fleming’s VARK model [15] , [16] (visual (V), auditory (A), read/write (R), and kinesthetic (K)). Since information delivered in a visual format, either by itself or in combination with other formats was preferred by the majority of participants, we also investigate some of the particular characteristics of video tutorials that participants prefer, such as their length;
    参与者在学习新主题时更喜欢使用的资源格式,使用 Fleming 的 VARK 模型 [15] , [16] (视觉 (V)、听觉 (A)、阅读/书写 (R) 和动觉 (K))描述的维度进行分类。由于大多数参与者更喜欢以视觉格式传递的信息,无论是单独还是与其他格式结合使用,我们还调查了参与者喜欢的视频教程的一些特定特征,例如它们的长度;

  2. the information seeking behavior of developers and their preferred sources of information during learning;
    开发人员的信息寻求行为及其在学习过程中偏好的信息来源;

  3. the aspects participants like and dislike about using current online resources, as well as the ways in which participants would like to see online resources improved to better support their learning activities; and
    参与者喜欢和不喜欢使用当前在线资源的方面,以及参与者希望看到在线资源改进以更好地支持他们的学习活动的方式;

  4. the non-technical topics and skills that participants would like to be able to learn more about online.
    参与者希望能够在线学习更多的非技术性话题和技能。

SECTION II.  第二部分。

Background and previous work
背景和以前的工作

In this paper, we categorize the medium (or format) of the information sources that our survey participants prefer to use for their learning tasks using Fleming’s VARK model [15] , [16] . The four learning modalities in Fleming’s model are: visual (V), auditory (A), read/write (R), and kinesthetic (K). While Fleming’s VARK model is no longer used to explain learning styles 1 , we found the four categories it defines useful in classifying the format of the learning resources respondents prefer to use. Therefore, our use of the VARK model does not refer to or support the concept of learning styles; we merely use it to describe preferences in resource formats, as follows. The visual format refers to having a preference for the depiction of information in illustrations, diagrams, pictures, maps, charts, graphs, and videos. The auditory or aural format refers to a preference for information that is “heard or spoken” (lectures, discussions, or tapes). The read/write preference is for information displayed as words and it emphasizes text-based input and output – reading and writing in all its forms but especially resources like slides, manuals, reports, essays, articles, and references. Tactile/kinesthetic is a preference for experience-centered learning, which involves actively performing an action, project, or experiment. In our survey, we also allow participants to select more than one type of format, to choose combinations of formats, and even to specify their own type of format.
在本文中,我们使用 Fleming 的 VARK 模型 [15] [16] 对调查参与者在学习任务中偏好的信息来源的媒介(或格式)进行分类。Fleming 模型中的四种学习模式是:视觉(V)、听觉(A)、阅读/书写(R)和动觉(K)。虽然 Fleming 的 VARK 模型不再用于解释学习风格 1 ,但我们发现它定义的四个类别在分类受访者偏好的学习资源格式时非常有用。因此,我们使用 VARK 模型并不是指或支持学习风格的概念;我们仅用它来描述资源格式的偏好,如下所示。视觉格式指的是偏好通过插图、图表、图片、地图、图表、图形和视频来展示信息。听觉或听觉格式指的是偏好“听到或说出”的信息(讲座、讨论或录音)。 阅读/书写偏好是指以文字形式显示的信息,强调基于文本的输入和输出——以各种形式的阅读和写作,尤其是幻灯片、手册、报告、论文、文章和参考资料等资源。触觉/动觉偏好是指以体验为中心的学习方式,涉及积极地执行某个动作、项目或实验。在我们的调查中,我们还允许参与者选择多种格式,选择格式的组合,甚至指定他们自己的格式类型。

In the following, we discuss related work about (i) studies on e-learning (ii) developers’ habits when seeking information on the Web, and (iii) work aimed at analyzing software development-related Web resources with the aim of better supporting developers.
在下文中,我们讨论了相关工作,包括 (i) 关于电子学习的研究 (ii) 开发人员在网上寻找信息的习惯,以及 (iii) 旨在分析与软件开发相关的网络资源以更好地支持开发人员的工作。

A. Studies on E-Learning and Internet Usage for Learning
A. 关于电子学习和互联网学习使用的研究

There have been numerous studies investigating the principles of e-learning and the habits exhibited by people, and in particular students, using online and electronic resources for learning. The studies have generally found that, whether they need only the definition of a term or a detailed discussion, for a large number of students, the search for information begins with search engines such as Google and free online encyclopedias and dictionaries such as Wikipedia [3] , [7] , [13] , [20] , [27] . Matusiak [38] and Uçak [71] concluded that students use search engines more frequently because of the ease of access and their satisfaction with the information search engines provide. Other studies have also found that personal websites, blogs, and discussion forums dedicated to teaching issues are often the most sparsely used electronic resources for learning purposes [9] , [27] , [28] , [77] .
有许多研究调查了电子学习的原理以及人们,特别是学生,使用在线和电子资源进行学习时表现出的习惯。研究普遍发现,无论他们只需要一个术语的定义还是详细的讨论,对于大量学生来说,信息搜索通常从搜索引擎如 Google 和免费在线百科全书和词典如 Wikipedia [3] [7] [13] [20] [27] 开始。Matusiak [38] 和 Uçak [71] 认为学生更频繁地使用搜索引擎是因为访问方便以及对搜索引擎提供的信息感到满意。其他研究还发现,个人网站、博客和专门讨论教学问题的论坛通常是用于学习目的的最少使用的电子资源 [9] [27] [28] [77]

Dukic et al. [13] found that the level and field of study of students are strong predictors of differences in the frequency of use of electronic resources for learning and that students slightly preferred electronic over printed resources. Other studies have also found that students are more inclined to use electronic resources [25] , [30] , [42] , [71] . The students appreciated the ease of use, search, and access to information [31] , [53] , [71] , recency of information [42] , [71] , availability of resources [60] , access to a wide range of information [42] , [53] , and so on. However, there are still many students who prefer printed sources [57] , [61] , even in technical fields [29] .
Dukic 等人 [13] 发现,学生的学习水平和专业领域是预测电子资源使用频率差异的重要因素,学生稍微偏好电子资源而非印刷资源。其他研究也发现,学生更倾向于使用电子资源 [25] [30] [42] [71] 。学生们欣赏电子资源的易用性、搜索和信息获取的便捷性 [31] [53] [71] ,信息的时效性 [42] [71] ,资源的可用性 [60] ,获取广泛信息的能力 [42] [53] ,等等。然而,仍有许多学生更喜欢印刷资源 [57] [61] ,即使在技术领域也是如此 [29]

Our study complements previous research in this area. In particular, RQ 1 indicates the extent to which computer science students and professionals prefer different resource formats (a combination of visual and auditory resources was particularly preferred), while results of RQ 2 indicate what are the most commonly used sources of information (written tutorials were preferred in particular).
我们的研究补充了该领域之前的研究。特别是,RQ 1 表明计算机科学学生和专业人士在多大程度上偏好不同的资源格式(特别偏好视觉和听觉资源的组合),而 RQ 2 的结果表明最常用的信息来源是什么(特别偏好书面教程)。

B. Developer Information Needs on the Web
B. 开发者在网络上的信息需求

While previous work has studied how open source projects use mailing lists for information seeking purposes [59] , nowadays developers use online Web resources quite often, as described in several studies. Gallardo-Valencia and Sim [18] provide a high-level categorization of programmers’ information needs that they address by using online resources. They find that one of the most common reasons for consulting Web resources is a lack of know-how, i.e., a lack of the necessary technical information to either implement a source code fragment, use a tool or command, or configure and deploy a software system. Numerous other studies [1] , [12] , [17] , [21] , [22] , [36] , [58] , [62] , [67] , [75] have also identified that developers frequently follow online resources as a reference material to implement their own solution. Programmers also resort to the Web when they lack the “know-that”, which involves a deeper understanding of a programming topic, rather than a quick solution for a specific programming task [1] , [12] , [17] , [18] , [67] . Common programming activities in this category are learning a new programming language from scratch [18] , learning about the functionality of an API [5] , and learning about new or advanced features of a programming language the developer is already familiar with [75] . Another category of information needs is “lack of recall”, where developers deliberately decide to forget the details of implementations they know well, knowing they can use the Web later as a mnemonic tool to remember the details ( e.g., the nature of a method or its parameters [18] and boilerplate code [5] ). Finally, troubleshooting covers information needs related to understanding the causes of errors and finding solutions to them [36] , [67] , [75] [18] .
尽管之前的研究已经探讨了开源项目如何使用邮件列表来寻求信息 [59] ,但如今开发者经常使用在线网络资源,如几项研究中所描述的那样。Gallardo-Valencia 和 Sim [18] 提供了一个程序员信息需求的高级分类,他们通过使用在线资源来解决这些需求。他们发现,咨询网络资源的最常见原因之一是缺乏专业知识,即缺乏必要的技术信息来实现源代码片段、使用工具或命令,或配置和部署软件系统。许多其他研究 [1] [12] [17] [21] [22] [36] [58] [62] [67] [75] 也发现开发者经常将在线资源作为参考材料来实现他们自己的解决方案。当程序员缺乏“知道什么”时,他们也会求助于网络,这涉及对编程主题的更深入理解,而不是对特定编程任务的快速解决方案 [1] [12] [17] [18] [67] 。 在这个类别中常见的编程活动是从头开始学习一种新的编程语言 [18] ,了解 API 的功能 [5] ,以及学习开发人员已经熟悉的编程语言的新功能或高级功能 [75] 。另一类信息需求是“缺乏回忆”,开发人员故意决定忘记他们熟悉的实现细节,知道他们可以稍后使用网络作为记忆工具来记住细节(例如,方法的性质或其参数 [18] 和样板代码 [5] )。最后,故障排除涵盖了与理解错误原因和找到解决方案相关的信息需求 [36] [67] [75] [18]

Additional information needs described by previous work are comparing different software components or pieces of code [18] , [67] , and retrieving pieces of source code [1] , [5] , [21] , [22] , [58] , [62] , [75] , or entire libraries or software systems [18] to be reused as-is.
先前工作描述的附加信息需求是比较不同的软件组件或代码片段 [18] [67] ,以及检索代码片段 [1] [5] [21] [22] [58] [62] [75] ,或整个库或软件系统 [18] 以原样重用。

Our study complements this previous work by focusing on the way in which developers satisfy these information needs (RQ 1 ), on ways in which existing resources currently support or could better support them (RQ 3 ), and also uncovers information needs concerning soft-skills not investigated in previous work (RQ 4 ).
我们的研究通过关注开发人员满足这些信息需求的方式(RQ 1 ),现有资源目前支持或可以更好地支持这些需求的方式(RQ 3 ),并揭示了先前工作中未调查的关于软技能的信息需求(RQ 4 ),来补充之前的工作。

C. Analyzing Software Engineering Web Resources
C. 分析软件工程网络资源

Previous literature investigated which online resources developers prefer to use. They found that developers often consult API specifications and source code documentation [10] , [12] , [21] , [36] , [65] , forums [10] , [36] , [65] , text-based tutorials [10] , [36] , [65] , code hosting sites [1] , [36] , [62] , technical blogs and Q&A sites [10] , [36] , [67] , and online training courses [10] . Several studies also found that common search engines ( e.g., Google) are very popular to find relevant online resources [5] , [10] , [11] , [26] , [65] , even overcoming in some cases specialized tools such as code-centered search engines ( e.g., Koders) and software repositories ( e.g., Source-Forge) [62] .
以往的文献调查了开发人员喜欢使用哪些在线资源。他们发现开发人员经常查阅 API 规范和源代码文档 [10] [12] [21] [36] [65] ,论坛 [10] [36] [65] ,基于文本的教程 [10] [36] [65] ,代码托管网站 [1] [36] [62] ,技术博客和问答网站 [10] [36] [67] ,以及在线培训课程 [10] 。几项研究还发现,常见的搜索引擎(例如,Google)在查找相关在线资源时非常受欢迎 [5] [10] [11] [26] [65] ,甚至在某些情况下超过了专门的工具,如以代码为中心的搜索引擎(例如,Koders)和软件库(例如,Source-Forge) [62]

Based on these observations, several studies have recognized the importance of analyzing, understanding, leveraging, and improving online software documentation [19] , [35] , [43] , [47] , [51] , [56] , [68] . In particular, researchers have been investigating online documentation sources such as technical blogs and micro-blogs [4] , [23] , [24] , [45] , [63] , Q&A sites such as Stack Overflow [44] , [48] , [50] , [69] , [74] , developer forums [19] , [39] , [50] , [70] , API documentation [43] , [66] , [68] , [73] , [78] , and crowd-based software development videos [37] , [40] , [41] , [46] , [49] , [76] .
基于这些观察,几项研究已经认识到分析、理解、利用和改进在线软件文档的重要性 [19] [35] [43] [47] [51] [56] [68] 。特别是,研究人员一直在调查在线文档来源,如技术博客和微博 [4] [23] [24] [45] [63] ,问答网站如 Stack Overflow [44] [48] [50] [69] [74] ,开发者论坛 [19] [39] [50] [70] ,API 文档 [43] [66] [68] [73] [78] ,以及基于众包的软件开发视频 [37] [40] [41] [46] [49] [76]

A recent work by Kim and Ko [33] has analyzed a diverse set of online coding tutorials in various formats from a pedagogical point of view. They found that the tutorials taught similar content and organized the material bottom-up. They also found that only a few coding tutorials were adaptive to the learners’ prior coding knowledge, and also only a few informed learners were able to transfer and apply the knowledge they learned.
Kim 和 Ko [33] 最近的一项工作从教学的角度分析了各种格式的在线编码教程。他们发现这些教程教授了相似的内容,并且是自下而上地组织材料的。他们还发现,只有少数编码教程能够适应学习者的先前编码知识,只有少数知情的学习者能够转移和应用他们所学的知识。

With respect to related work mentioned above, our study provides — across all research questions and as discussed in Section IV — insights on how to improve automatic analysis of online resources for developers and the inception of better recommender systems.
关于上述相关工作,我们的研究在所有研究问题中(如在 Section IV 中讨论的那样)提供了关于如何改进开发人员在线资源的自动分析和更好的推荐系统的见解。

SECTION III.  第三部分。

Methodology  方法论

The goal of our empirical study is to investigate the learning habits and preferences of computer science students and programming practitioners, with the purpose of understanding how such learning benefits from the usage of online resources. The study context consists of a survey delivered to 205 participants. The survey was forwarded by one of the authors through Facebook among a large (over 1,000 members), but closed group of University students, and also distributed publicly on Twitter among professional software developers. Due to the social media connections of this author to some high-profile software developers, the survey was adopted and redistributed to a large network of experienced software practitioners on social media. The perspective of this study is that of researchers, interested in understanding the influence of online resources in the learning activities of CS students and experienced practitioners. The long-term goals, as a consequence of the study results, are twofold: (a) to provide better understanding for researchers on the usage, limitations, and advantages of online resources for learning CS concepts, (b) to provide learning support for developers and CS students — e.g., through better recommender systems — by exploiting the resources practitioners access most frequently and by keeping into account their preferred resource format.
我们实证研究的目标是调查计算机科学学生和编程从业者的学习习惯和偏好,目的是了解这种学习如何从使用在线资源中受益。研究背景包括向 205 名参与者发放的调查问卷。调查问卷由其中一位作者通过 Facebook 在一个大型(超过 1000 名成员)但封闭的大学生群体中转发,并在 Twitter 上公开分发给专业软件开发人员。由于这位作者与一些高知名度软件开发人员的社交媒体联系,调查被采用并重新分发给社交媒体上经验丰富的软件从业者的大型网络。本研究的视角是研究人员,旨在了解在线资源对计算机科学学生和经验丰富的从业者学习活动的影响。 由于研究结果,长期目标有两个:(a) 为研究人员提供更好的理解,了解在线资源在学习计算机科学概念中的使用、限制和优势,(b) 为开发人员和计算机科学学生提供学习支持——例如,通过更好的推荐系统——利用从业者最常访问的资源,并考虑他们偏好的资源格式。

Given this context and the participants in our study, we address four different research questions. First and foremost, we are interested in achieving a broad understanding of the participants’ preferred resource format for what concerns learning programming concepts. To this aim, our first research question is:
鉴于这一背景和我们研究中的参与者,我们解决了四个不同的研究问题。首先,我们对参与者在学习编程概念时所偏好的资源格式有一个广泛的了解。为此,我们的第一个研究问题是:

RQ 1 - Resource format What resource format/medium do the survey participants prefer?
RQ 1 - 资源格式 调查参与者更喜欢哪种资源格式/媒介?

More specifically, we are interested in knowing which formats (categorized using Fleming’s VARK model [15] , [16] as V isual, A uditory, R ead/write, and K inesthetic) the participants prefer. Since people can prefer more than one format, we allowed participants to select more than one resource format and even to specify their preferred formats and combinations.
更具体地说,我们感兴趣的是了解参与者更喜欢哪种格式(使用 Fleming 的 VARK 模型 [15] [16] 分类为视觉、听觉、读/写和动觉)。由于人们可以喜欢多种格式,我们允许参与者选择多种资源格式,甚至指定他们喜欢的格式和组合。

Furthermore, our study focuses on some details regarding video tutorials, as they have been widely used [32] to deliver educational material, especially for distance learning and they are also a resource on the rise in popularity among developers [37] , [49] . In particular, we are interested in learning about the participants’ preferred video tutorial format — i.e., whether they prefer a single video on a comprehensive topic, or whether they find it easier to browse multiple, smaller videos — and about their preferred length for videos.
此外,我们的研究重点关注视频教程的一些细节,因为它们已被广泛用于 [32] 传递教育材料,特别是远程学习,它们也是开发者中越来越受欢迎的资源 [37] [49] 。特别是,我们对参与者偏好的视频教程格式感兴趣——即他们是更喜欢关于综合主题的单个视频,还是觉得浏览多个较小的视频更容易——以及他们对视频长度的偏好。

We then investigate in more in detail how developers seek the information necessary to understand programming concepts — e.g., from official documentation, lectures, video tutorials, or other online resources — and possibly interleave such resources with programming tasks. Therefore, our second research question is:
我们接着更详细地调查开发人员如何寻找理解编程概念所需的信息——例如,从官方文档、讲座、视频教程或其他在线资源——并可能将这些资源与编程任务交错使用。因此,我们的第二个研究问题是:

RQ 2 - Learning preferences: What are the participants’ preferences when learning programming topics?
RQ 2 - 学习偏好:参与者在学习编程主题时的偏好是什么?

More specifically, we identify the sources of information participants use to learn new topics in programming, as well as the format they prefer when they try to master a new programming language. Therefore, we address the following sub-question of RQ 2 :
更具体地说,我们确定了参与者用来学习编程新主题的信息来源,以及他们在尝试掌握新编程语言时所偏好的格式。因此,我们解决了 RQ 2 的以下子问题:

RQ 2.1 When they need information about a new topic, what are the participants’ most common information seeking habits?
RQ 2.1 当他们需要了解一个新话题时,参与者最常见的信息搜寻习惯是什么?

In other words, we ask participants how they find relevant information when they learn a new programming topic. More specifically, we ask whether they access online University lectures, written tutorials, video tutorials, official programming language/technology documentation, or any other material found on the Web. Additionally, we also collect answers about the most common websites they use to get such information.
换句话说,我们询问参与者在学习新的编程主题时如何找到相关信息。更具体地说,我们询问他们是否访问在线大学讲座、书面教程、视频教程、官方编程语言/技术文档或在网上找到的任何其他材料。此外,我们还收集他们获取此类信息时最常用的网站的答案。

There are also different ways programming-related content could be illustrated. In particular, one could use a specific programming language or generic pseudo-code for this purpose. Therefore, we also address the following subquestion:
还有不同的方法可以展示与编程相关的内容。特别是,可以使用特定的编程语言或通用的伪代码来实现这一目的。因此,我们还要解决以下子问题:

RQ 2.2 What are the participants’ language preferences when learning new programming-specific content?
RQ 2.2 学习新的编程特定内容时,参与者的语言偏好是什么?

Specifically, given a programmer familiar with a particular programming language that wants to learn a (not language-specific) programming concept, we ask whether she prefers a tutorial prepared using their preferred language, any programming language, or just pseudo-code.
具体来说,考虑到一个熟悉特定编程语言的程序员想要学习一个(非语言特定的)编程概念,我们询问她是否更喜欢使用她喜欢的语言、任何编程语言或仅仅伪代码编写的教程。

For RQ 1 and RQ 2 , we use multiple-choice questions. In some cases, the questions allow more than one possible option ( i.e., multiple answers were allowed), or even to type in an answer that was not initially included in the set of choices.
对于 RQ 1 和 RQ 2 ,我们使用选择题。在某些情况下,问题允许多个可能的选项(即,允许多个答案),甚至可以输入最初未包含在选项中的答案。

Then, while RQ 1 aims at achieving a broad understanding about the preferred resource formats , and RQ 2 at investigating what resources are being used and how, our third research question aims at understanding the why :
然后,当 RQ 1 旨在广泛了解首选的资源格式时,RQ 2 旨在调查正在使用哪些资源以及如何使用,我们的第三个研究问题旨在理解原因:

RQ 3 - likes, dislikes, and wishes about online resources: What are the likes, dislikes, and wishes of the participants when using online resources to learn programming/computer science topics?
RQ 3 - 关于在线资源的喜欢、不喜欢和愿望:参与者在使用在线资源学习编程/计算机科学主题时,有哪些喜欢、不喜欢和愿望?

We asked the participants about the things they like and dislike about existing online resources, as well as the things they wish to see in online resources. Specifically, we ask the following three open-ended questions:
我们询问了参与者关于他们喜欢和不喜欢现有在线资源的内容,以及他们希望在在线资源中看到的内容。具体来说,我们提出了以下三个开放性问题:

  1. “What are the things you like about existing resources for learning programming concepts?”
    “你喜欢现有的学习编程概念资源的哪些方面?”

  2. “What are the things you dislike about existing resources for learning programming concepts?”
    “你对现有的编程概念学习资源有哪些不喜欢的地方?”

  3. “What are the things you wish existing resources for learning programming concepts provided?”
    “你希望现有的编程概念学习资源提供哪些内容?”

Although online resources play an important role in providing valuable technical information to our participants and to software developers in general, they can be also helpful when learning non-technical skills useful for a career in programming. We asked our participants about the soft-skills they would like to learn more about by using online resources. We consider two main categories of soft-skills. The first category describes skills that are more relevant in the workplace, and the second category relates to soft-skills associated with self-care and happiness as a programming practitioner. Therefore, our fourth research question is formulated as follows:
尽管在线资源在为我们的参与者和软件开发人员提供有价值的技术信息方面发挥着重要作用,但它们在学习对编程职业有用的非技术技能时也能提供帮助。我们询问了参与者他们希望通过在线资源学习哪些软技能。我们将软技能分为两个主要类别。第一类描述了在工作场所更为相关的技能,第二类则与作为编程从业者的自我关怀和幸福相关的软技能。因此,我们的第四个研究问题表述如下:

RQ 4 - Soft Skills: What are the soft-skills that participants would like to learn more about from online resources?
RQ 4 - 软技能:参与者希望从在线资源中了解哪些软技能?

In this respect, we asked a question about learning soft-skills relevant to the workplace, i.e., how to ask for help, how to approach inappropriate situations, how to negotiate a salary, etc. We then also asked a question about learning other soft-skills related to self-care and happiness, i.e., work-life balance, ways to remain healthy, how to deal with feeling exhausted or unhappy, etc. The participants could also type in other soft skills they would like to learn more about from online resources.
在这方面,我们提出了一个关于学习与工作场所相关的软技能的问题,即如何寻求帮助,如何处理不适当的情况,如何谈判薪水等。然后我们还提出了一个关于学习与自我照顾和幸福相关的其他软技能的问题,即工作与生活的平衡,保持健康的方法,如何应对疲惫或不开心的感觉等。参与者还可以输入他们希望从在线资源中了解更多的其他软技能。

To address RQ 1 , RQ 2 , and RQ 4 , we first classified the survey respondents into Students or Professionals according to their self-reported experience level. Then, for each group, we report (in form of bar charts and tables) the distributions of the provided answers, and discuss them. When analyzing results between students and professionals, we compare proportions among the two categories using a proportion test [14] with a significance level α = 5%. Specifically, we use the prop.test implementation available in R [52] .
为了解决 RQ 1 、RQ 2 和 RQ 4 ,我们首先根据调查受访者自报的经验水平将其分类为学生或专业人士。然后,对于每个组,我们以条形图和表格的形式报告提供答案的分布,并进行讨论。在分析学生和专业人士之间的结果时,我们使用显著性水平 α = 5% 的比例检验 [14] 比较两类之间的比例。具体来说,我们使用 R 中可用的 prop.test 实现 [52]

For what concerns RQ 3 , since the answers are open-ended, we adopted an open-coding process followed by a card sorting phase [64] . First, by reading the answers, one of the authors identified codes for each answer, using an online spreadsheet. After that, a second author revised the codes, with the aim of performing merging or splitting actions where needed, or of pointing cases for which the initial classification was questionable. Such unclear cases were discussed and resolved. Then, the first author went through the refined codes and grouped the answers across categories.
关于 RQ 3 ,由于答案是开放式的,我们采用了开放编码过程,然后进行了卡片分类阶段 [64] 。首先,通过阅读答案,其中一位作者使用在线电子表格为每个答案确定了代码。之后,第二位作者对代码进行了修订,目的是在需要的地方进行合并或拆分操作,或者指出初始分类有问题的情况。这些不明确的情况进行了讨论和解决。然后,第一位作者浏览了精炼后的代码,并将答案分组到各个类别中。

The 205 collected responses are available at https://bit.ly/2TS3LaY
205 份收集的回复可在 https://bit.ly/2TS3LaY 查看

SECTION IV.  第四节。

Results and discussion  结果与讨论

We obtained a total of 205 answers to the survey, which included a mix of computer science students and professional developers. In the following subsections, we first provide demographic information about the participants and then present and discuss the results of our survey, with the aim of answering our research questions.
我们总共获得了 205 份调查问卷的回答,其中包括计算机科学学生和专业开发人员的混合。在以下小节中,我们首先提供参与者的人口统计信息,然后展示和讨论我们的调查结果,旨在回答我们的研究问题。


Fig. 1
 - 

a) Gender and b) Age distribution of survey participants.
Fig. 1  图 1

a) Gender and b) Age distribution of survey participants.
a) 调查参与者的性别和 b) 年龄分布。

A. Demographics  A. 人口统计

As depicted in Fig. 1a , 17.5% of the respondents are female, whereas the 79% of the participants are male. We also obtained answers from 7 (3.4%) people that did not disclose their gender. Regarding age, we obtained a significant proportion of answers (36.1%) from participants older than 35, even though we had initially anticipated a large participation from younger professionals and college students due to the social networks used to distribute the survey. Then, most of our remaining answers come from participants that are between 18 and 24 years old (23.9%), between 25 and 30 years old (21%), and between 31 and 35 years old (16.1%). Fig. 1b presents the distribution of the participants’ age groups.
Fig. 1a 所示,17.5%的受访者是女性,而 79%的参与者是男性。我们还收到了 7 名(3.4%)未透露性别的人的回答。关于年龄,我们从 35 岁以上的参与者中获得了大量的回答(36.1%),尽管我们最初预计由于使用社交网络分发调查问卷,年轻专业人士和大学生的参与会更多。然后,我们的大多数剩余回答来自 18 至 24 岁(23.9%)、25 至 30 岁(21%)和 31 至 35 岁(16.1%)的参与者。 Fig. 1b 展示了参与者年龄组的分布。

Fig. 2 reports information about the participants’ highest degree attained, while their experience in CS is shown in Fig. 3 . Most of our participants (51.2%) completed a bachelor degree, whereas about a quarter (27.8%) of the respondents finished a graduate program. With the exception of four people (2%), all the remaining participants have completed high-school (19%).
Fig. 2 报告了参与者所获得的最高学位,而他们在计算机科学方面的经验显示在 Fig. 3 。我们的大多数参与者(51.2%)完成了学士学位,而大约四分之一(27.8%)的受访者完成了研究生课程。除四人(2%)外,其余所有参与者都完成了高中学业(19%)。


Fig. 2
 - 

Distribution of the highest educational degree obtained by survey participants.
Fig. 2  图 2

Distribution of the highest educational degree obtained by survey participants.
调查参与者所获得的最高学历分布。


Fig. 3
 - 

Distribution of the experience in CS expressed by survey participants.
Fig. 3  图 3

Distribution of the experience in CS expressed by survey participants.
调查参与者表达的计算机科学经验分布。

Regarding the experience in CS, we found that the majority of our participants (60.5%) consider themselves to have an Advanced level of knowledge/experience in the field (defined as “very experienced from years of work or projects beyond school”). 34.1% of the participants reported having an Intermediate level of expertise, mostly based on school projects (Intermediate was defined as “School experience or some professional/project experience”), while 4.9% were Beginners (i.e., having “No experience or minimal experience/familiarity” with CS). We note that one of our participants indicated never having considered getting into CS/programming so far; therefore, we did not consider his/her answers for the rest of the study. Based on these results, we categorized the 80 respondents that indicated a Beginner or Intermediate experience level as Students and the 124 respondents that reported to have an Advanced level of experience as Professionals. This is based on the fact that in the definitions used in our survey, Advanced was the only level containing significant experience “beyond school”.
关于计算机科学的经验,我们发现大多数参与者(60.5%)认为自己在该领域具有高级知识/经验(定义为“多年来在工作或项目中积累了丰富经验”)。34.1%的参与者报告说他们具有中级专业知识,主要基于学校项目(中级定义为“学校经验或一些专业/项目经验”),而 4.9%是初学者(即“没有经验或对计算机科学的熟悉程度很低”)。我们注意到有一位参与者表示从未考虑过进入计算机科学/编程领域,因此我们没有将他/她的回答纳入研究的其余部分。基于这些结果,我们将表示具有初学者或中级经验水平的 80 名受访者分类为学生,将报告具有高级经验水平的 124 名受访者分类为专业人士。这是基于我们调查中使用的定义,高级是唯一包含“超出学校”显著经验的级别。

B. RQ 1 - Resource format
B. RQ 1 - 资源格式

We asked our participants about their preferred format for information resources. In particular, we asked them whether they have any special preference in learning using resources whose content is mostly presented using visual elements ( e.g., illustrations, diagrams, pictures, etc.), auditory elements ( i.e., entirely spoken or with sounds), resources with only text, or learning with resources that require some sort of hands-on activity. We also considered combinations of styles, such as text and visual elements, a combination of visual and audio, etc. As explained in Section III , participants could provide multiple answers to this question, as well as adding options not included in our set of choices.
我们询问了参与者关于他们对信息资源的首选格式。特别是,我们问他们是否在使用主要以视觉元素(例如插图、图表、图片等)呈现内容的资源、听觉元素(即完全口述或带有声音)、仅有文本的资源,或需要某种动手活动的资源进行学习时有任何特殊偏好。我们还考虑了风格的组合,例如文本和视觉元素的组合、视觉和音频的组合等。如在 Section III 中所述,参与者可以对这个问题提供多个答案,并添加我们选择集中未包含的选项。

Fig. 4 overviews the distribution of the preferred reference formats for both groups of respondents. Most of the participants chose a combination of visual/auditory formats as their top preference (65% of Students, 55.6% of Professionals), with the visual format being the top single format preferred (25% of Students, 28.2% Professionals). A small number of respondents also expressed that auditory was one of their favorite formats (3.7% Students, 2.4% Professionals). In general, the very low preference for the auditory-only format is not surprising, considering the need for showing source code or diagrams when learning programming, as opposed to humanistic disciplines where a purely auditory style could be acceptable, if not expected. At the same time, it is also true that a purely visual delivery format might be less effective than when the images, presentations, or screencasts are accompanied by an explanatory voice, which may explain the preference for a combination of the two formats. Six participants (2.9%) indicated a preference for using resources with text (or text and visuals), while nine of them (4.4%) prefer to have some hands-on experience with online resources to facilitate their learning, which previous literature found to be particularly effective [34] . Moreover, some participants also reported their preferences when it comes to learning only by themselves, or as a part of a study group. In particular, 95 participants (51.2% of the Students and 43.5% of the Professionals) prefer to study/learn on their own, whereas other 26 participants (13.7% of the Students and 12.1% of the Professionals) expressed a preference to learn in a group. Overall, the proportion test did not indicate any statistically significant difference between students and professionals ( p -value=0.50).
Fig. 4 概述了两组受访者的首选参考格式的分布情况。大多数参与者选择了视觉/听觉格式的组合作为他们的首选(65%的学生,55.6%的专业人士),其中视觉格式是最受欢迎的单一格式(25%的学生,28.2%的专业人士)。少数受访者也表示听觉是他们最喜欢的格式之一(3.7%的学生,2.4%的专业人士)。总体而言,听觉唯一格式的偏好非常低并不令人惊讶,考虑到在学习编程时需要展示源代码或图表,而在人文学科中,纯听觉风格可能是可以接受的,甚至是预期的。同时,纯视觉传递格式可能不如图像、演示或屏幕录像伴随解释性声音时有效,这也可能解释了对两种格式组合的偏好。六名参与者(2.9%)表示偏好使用带有文本(或文本和视觉)的资源,而其中九名(4.4%)更喜欢通过在线资源进行一些实践操作来促进他们的学习,之前的文献发现这种方法特别有效 [34] 。此外,一些参与者还报告了他们在自学或作为学习小组的一部分时的偏好。特别是,95 名参与者(51.2%的学生和 43.5%的专业人士)更喜欢独自学习,而其他 26 名参与者(13.7%的学生和 12.1%的专业人士)表示更喜欢在小组中学习。总体而言,比例检验未显示学生和专业人士之间存在任何统计学上的显著差异(p 值=0.50)。


Fig. 4
 - 

Distribution of the preferred reference formats.
Fig. 4  图 4

Distribution of the preferred reference formats.
首选参考格式的分布。

We then asked the participants about their preferences when it comes to watching video tutorials. According to Fig. 5 , most of them (65.2%) prefer multiple short videos on a topic (61.2% of the Students and 67.7% of the Professionals), with only 16.2% of respondents preferring only one, longer video. Note that almost a fifth of the participants (18.7% of the Students and 18.5% of the Professionals) indicated that they do not use video tutorials to learn a new topic. Participants also expressed a preference mainly for short videos. As depicted in Fig. 6 , 55.9% of them (56.2% of the Students and 55.6% of the Professionals) prefer videos between 5 and 15 minutes long. For both Fig. 5 and Fig. 6 , there is no statistically significant difference between students and professionals ( p -value=0.47 and 0.46, respectively).
然后我们询问了参与者在观看视频教程时的偏好。根据 Fig. 5 ,大多数人(65.2%)更喜欢一个主题的多个短视频(61.2%的学生和 67.7%的专业人士),只有 16.2%的受访者更喜欢一个较长的视频。请注意,几乎五分之一的参与者(18.7%的学生和 18.5%的专业人士)表示他们不使用视频教程来学习新主题。参与者还表示主要偏好短视频。如 Fig. 6 所示,55.9%的人(56.2%的学生和 55.6%的专业人士)更喜欢时长在 5 到 15 分钟之间的视频。对于 Fig. 5 Fig. 6 ,学生和专业人士之间没有统计学上的显著差异(p 值分别为 0.47 和 0.46)。


Fig. 5
 - 

Distribution of preferred video formats: A) among students, and B) among professionals.
Fig. 5  图 5

Distribution of preferred video formats: A) among students, and B) among professionals.
首选视频格式的分布:A) 在学生中,B) 在专业人士中。


Fig. 6
 - 

Distribution of preferred video length: A) among students, and B) among professionals. Inf=“As many hours as it takes in one video”. No video=“I do not use video resources”.
Fig. 6  图 6

Distribution of preferred video length: A) among students, and B) among professionals. Inf=“As many hours as it takes in one video”. No video=“I do not use video resources”.
视频长度偏好分布:A) 在学生中,B) 在专业人士中。Inf=“一个视频中所需的小时数”。无视频=“我不使用视频资源”。

Overall, the obtained results reflect what was found in previous work by MacLeod et al. [37] , but also by Ponzanelli et al. [49] who proposed an approach to split video tutorials in small, cohesive fragments and received from evaluators insights about a preference for shorter videos.
总体而言,所得结果反映了 MacLeod 等人 [37] 之前工作的发现,同时也反映了 Ponzanelli 等人 [49] 提出的一种将视频教程分割成小而连贯片段的方法,并从评估者那里得到了关于偏好较短视频的见解。

Summary for RQ 1 : Participants largely prefer a reference format that involves a combination of visual and auditory stimuli. When it comes to using video tutorials, participants prefer short videos (max. 15 minutes long).
RQ 1 的总结:参与者大多偏好包含视觉和听觉刺激相结合的参考格式。对于使用视频教程,参与者更喜欢短视频(最长 15 分钟)。

C. RQ 2 - Learning preferences
C. RQ 2 - 学习偏好

As explained in Section III , we asked our participants — again using multiple-choice questions with the possibility of specifying further options — about their habits and preferences when they need to search for information for learning programming. In the following, we discuss results for the two sub-questions, related to their information seeking habits (RQ 2.1 ), and their preferences when learning a programming-specific or general CS concept (RQ 2.2 ).
正如在 Section III 中解释的那样,我们询问了参与者——再次使用多项选择题并有可能指定其他选项——关于他们在需要搜索编程学习信息时的习惯和偏好。接下来,我们讨论与他们的信息搜索习惯(RQ 2.1 )和学习编程特定或一般计算机科学概念时的偏好(RQ 2.2 )相关的两个子问题的结果。

TABLE I Most common sources of information. The number in parenthesis indicates the percentage of participants in the column that mentioned the resource in their answers.
表 I 最常见的信息来源。括号中的数字表示在回答中提到该资源的参与者百分比。
Table I
- 

Most common sources of information. The number in parenthesis indicates the percentage of participants in the column that mentioned the resource in their answers.

RQ 2.1 - Information seeking habits
RQ 2.1 - 信息寻求习惯

According to Table I , written tutorials and documentation/textbooks are the most common sources of information that our participants use to learn about programming, followed by video tutorials and university lectures. Also, we observe that 18.2% more people (4.9% more Students and 27.6% more Professionals) chose written tutorials compared to documentation or textbooks. As detailed in the results of RQ 3 , the difference between these two is partially explained by the format used in written tutorials to present the content; in general, these are broken down into small topics, use a conversational language, contain many source code examples, and sometimes include a Q&A section to provide feedback or ask further questions. Note that, even though a written tutorial and a piece of documentation or a textbook can explain the same programming topic, the style used in each one can impact a user’s engagement.
根据 Table I ,书面教程和文档/教科书是我们参与者用来学习编程的最常见信息来源,其次是视频教程和大学讲座。此外,我们观察到选择书面教程的人比选择文档或教科书的人多 18.2%(学生多 4.9%,专业人士多 27.6%)。如 RQ 3 的结果所详述,这两者之间的差异部分可以通过书面教程中使用的内容呈现格式来解释;一般来说,这些教程被分解成小主题,使用对话语言,包含许多源代码示例,有时还包括问答部分以提供反馈或提出进一步问题。请注意,尽管书面教程和文档或教科书可以解释相同的编程主题,但每种风格的使用可能会影响用户的参与度。

We also found a clear difference in the participants’ preferences between video tutorials and university video lectures. Even though both types of resources exercise the visual and auditory learning styles at the same time, 136% more participants (44.4% more Students and 260% more Professionals) included video tutorials in their preferences compared to university video lectures. Again, this can be explained by the way video tutorials structure and present relevant content ( e.g., short videos in sequence, conversational style) compared to university video lectures. Unsurprisingly, the difference concerning University video lectures is statistically significant ( p -value=0.015), as these were mainly considered useful by students rather than by professionals.
我们还发现参与者在视频教程和大学视频讲座之间的偏好存在明显差异。尽管这两种资源同时运用了视觉和听觉学习方式,但有 136%更多的参与者(学生多 44.4%,专业人士多 260%)在偏好中选择了视频教程,而不是大学视频讲座。同样,这可以通过视频教程在结构和呈现相关内容(例如,连续的短视频,对话风格)方面的方式来解释,而不是大学视频讲座。不出所料,关于大学视频讲座的差异在统计上是显著的(p 值=0.015),因为这些主要被学生认为是有用的,而不是专业人士。

Regarding the most common websites participants use to get information, we found that search engines ( e.g., Google) are the most common answer, followed by StackOverflow ( i.e., Q&A site), YouTube ( i.e., a video share website), GitHub ( i.e., a hosting service for version control of projects), Code Academy and Pluralsight ( i.e., online training platform based on videos), and Mozilla Dev Network (online documentation).
关于参与者获取信息时最常用的网站,我们发现搜索引擎(例如,Google)是最常见的答案,其次是 StackOverflow(即问答网站)、YouTube(即视频分享网站)、GitHub(即项目版本控制的托管服务)、Code Academy 和 Pluralsight(即基于视频的在线培训平台),以及 Mozilla Dev Network(在线文档)。

RQ 2.2 - Preferences when learning a programming-specific or CS concept
RQ 2.2 - 学习编程特定或计算机科学概念时的偏好

Table II presents the strategies that our survey participants use to learn a new programming-specific concept. In general, respondents prefer tutorials and examples that use either any programming language or pseudo code to learn CS concepts. While the use of programming languages is preferred (35.8%), only 27.9% would like to see examples using a particular programming language of their choice. In this case, we did not find any statistically significant difference between students and professionals ( p -value=0.93).
Table II 展示了我们的调查参与者用来学习新的编程特定概念的策略。总体而言,受访者更喜欢使用任何编程语言或伪代码的教程和示例来学习计算机科学概念。虽然更喜欢使用编程语言(35.8%),但只有 27.9%的人希望看到使用他们选择的特定编程语言的示例。在这种情况下,我们没有发现学生和专业人士之间有任何统计学上显著的差异(p 值=0.93)。

TABLE II Strategies used to learn a new programming or CS topic. The number in parenthesis indicates the percentage of participants in the column that mentioned the strategy in their answers.
表二 学习新的编程或计算机科学主题所使用的策略。括号中的数字表示该列中提到该策略的参与者的百分比。
Table II
- 

Strategies used to learn a new programming or CS topic. The number in parenthesis indicates the percentage of participants in the column that mentioned the strategy in their answers.

Since we did not collect additional information about the cases in which the participants prefer to use pseudo-code instead of a programming language, we conjecture that they use pseudo-code or any programming language to better understand high-level CS concepts ( e.g. , algorithms or data structures) rather than specific programming technologies. Moreover, some of the participants (5.4%) prefer not seeing any code at all when learning a programming or CS concept, which offers some evidence of the presence of different learning preferences among our respondents. For those exceptional cases, our assumption is that these participants use textual or visual elements ( e.g. , text in prose or diagrams) rather than code to better understand CS concepts.
由于我们没有收集关于参与者在何种情况下更喜欢使用伪代码而不是编程语言的额外信息,我们推测他们使用伪代码或任何编程语言是为了更好地理解高层次的计算机科学概念(例如,算法或数据结构),而不是具体的编程技术。此外,一些参与者(5.4%)在学习编程或计算机科学概念时更喜欢完全不看任何代码,这提供了一些证据,表明我们的受访者中存在不同的学习偏好。对于这些特殊情况,我们的假设是,这些参与者使用文本或视觉元素(例如,散文中的文字或图表)而不是代码来更好地理解计算机科学概念。

Summary for RQ 2 : Participants prefer to seek information from text-based resources more than from video-based ones, with short content and conversational style positively impacting their engagement. Finally, participants feel they can learn CS/programming from examples written in any programming language, not just the one they prefer.
RQ 2 的总结:参与者更喜欢从基于文本的资源中获取信息,而不是基于视频的资源。简短的内容和对话风格对他们的参与度有积极影响。最后,参与者认为他们可以从用任何编程语言编写的示例中学习计算机科学/编程,而不仅仅是他们喜欢的那种语言。

D. RQ 3 - likes, dislikes, and wishes about online resources
D. RQ 3 - 对在线资源的喜欢、不喜欢和愿望

In the following, we report and discuss the result of the card sorting we performed to address RQ 3 . For each category, we report the number of people that mentioned it in their answers in parenthesis, we mention (in bold face ) the codes we grouped within that category, and report some examples of the collected answers.
在下文中,我们报告并讨论我们为解决 RQ 3 而进行的卡片分类结果。对于每个类别,我们报告在回答中提到该类别的人数(括号内),我们提到(用粗体)我们在该类别中分组的代码,并报告一些收集的答案示例。

1) Likes: In the following, we describe the main reasons why participants like to learn CS/programming from online resources.
1) 喜欢:在下文中,我们描述了参与者喜欢从在线资源学习计算机科学/编程的主要原因。

General valuable features (133 participants): The most frequent aspects that participants highlighted about online resources were their diversity , abundance , and availability , the latter particularly characterized by the fact that online resources can be accessed for free . Examples of answers include “Good diversity in choices”, or “(…) almost any topic can be found online”.
一般有价值的特点(133 名参与者):参与者最常提到的在线资源的特点是其多样性、丰富性和可用性,尤其是在线资源可以免费访问这一特点。回答的例子包括“选择的多样性很好”或“(……)几乎任何主题都可以在网上找到”。

Good content (109 participants): Respondents included different facets of what they consider good quality online content. Additionally to the diversity and completeness of topics, participants also highlighted the effectiveness of online resources at offering answers to specific problems and information at variable levels of granularity , ranging from high-level descriptions to very detailed information about a technology. Additionally, they appreciate the available knowledge provided by experts , which in some cases can bring different perspectives to a single topic. Finally, respondents expressed that a good quality online resource usually is up-to-date and contains code examples . Examples of answers in this category are “(…) being able to ask specific questions, (…)”, or “Worked examples and explanations (sic) of the examples to put them in context”.
好的内容(109 名参与者):受访者包括了他们认为高质量在线内容的不同方面。除了主题的多样性和完整性外,参与者还强调了在线资源在提供特定问题的答案和不同粒度级别的信息方面的有效性,从高级描述到关于某项技术的非常详细的信息。此外,他们还欣赏专家提供的可用知识,这在某些情况下可以为单一主题带来不同的视角。最后,受访者表示,高质量的在线资源通常是最新的,并包含代码示例。此类别中的答案示例包括“(……)能够提出具体问题,(……)”或“工作示例和示例的解释(原文如此),以将它们置于上下文中”。

Nice format (34 participants): A minority of respondents like the way online resources can deliver information using different formats or mediums, such as books, tutorials, university lectures, etc. In some cases, they indicate preference for interactive formats, or formats that follow some sort of structure , like the ones found in online courses. Finally, they are also inclined for formats that can deliver information in a quick , friendly , and concise manner. Example answers: “(…) content available for free in many formats, some even interactive (codecademy.com)”, “usually in written form, most are short enough to read in one sitting”.
不错的格式(34 名参与者):少数受访者喜欢在线资源可以通过不同的格式或媒介传递信息,例如书籍、教程、大学讲座等。在某些情况下,他们表示更喜欢互动格式,或者遵循某种结构的格式,比如在线课程中找到的那些。最后,他们也倾向于能够以快速、友好和简明的方式传递信息的格式。示例回答:“(……)内容以多种格式免费提供,有些甚至是互动的(codecademy.com)”,“通常是书面形式,大多数都足够简短,可以一次读完”。

Easily Searchable (26 participants): Despite the general issues about searching in online resources (see dislikes), a few participants referred to the existence of search engines and indexes that significantly improve the searchability of the online resources, making them easy to find . Examples of answers are: “High searchability for most specific problems”, or “Google has indexed them. They are discoverable”.
易于搜索(26 名参与者):尽管在在线资源搜索方面存在一些普遍问题(见不喜欢的部分),但有些参与者提到了搜索引擎和索引的存在,这显著提高了在线资源的可搜索性,使其易于找到。回答示例包括:“大多数具体问题的高可搜索性”或“谷歌已对其进行索引。它们是可发现的”。

Collateral Advantages (15 participants): Participants also mentioned additional advantages or valuable elements that appear when using online resources. The most frequent value in this category is the ability to access content voluntarily shared by others, the ability to share their own content, or to bring feedback to existing resources. These relationships then may lead to the creation of communities or collaborative efforts to create online content. Examples of answers: “Established communities can provide answers in a light hearted and fast moving and educational way”, or “Available freely and open to community suggestions”.
附带优势(15 名参与者):参与者还提到了使用在线资源时出现的额外优势或有价值的元素。该类别中最常见的价值是能够访问他人自愿分享的内容、分享自己的内容或对现有资源提供反馈。这些关系随后可能会导致社区的创建或协作努力来创建在线内容。答案示例:“已建立的社区可以以轻松、快速和教育的方式提供答案”,或“免费提供并开放社区建议”。

2) Dislikes: In the following, we describe the main limitations participants perceived in using online resources to learn CS/programming.
2) 不喜欢:在下文中,我们描述了参与者在使用在线资源学习计算机科学/编程时感知到的主要限制。

Low quality (75 participants): Although, as mentioned above, the majority of participants found the online content valuable and, in general, of good quality, there were also some complaints in this respect. In particular, there is an inherent difficulty with assessing the quality of the results of a search for online resources, leading to situations in which the participants have to go over several low quality resources before finding the one that fulfills their requirements. The quality of an online resource, according to participants, is diminished by several factors: its content is outdated , it is a copy of another online resource, lack of examples or presence of bad code examples , lack of proper structure , lack of related resources , among others. Examples of answers include: “Some tutorials start strong but end very poorly and skip code sections which can confuse beginners or someone new to the concept”, or “Refusal or inability to provide exact solutions even when 95% of people have the exact same issue”.
低质量(75 名参与者):虽然如上所述,大多数参与者认为在线内容有价值且总体质量良好,但在这方面也有一些投诉。特别是,评估在线资源搜索结果的质量存在固有困难,导致参与者在找到满足其要求的资源之前必须浏览多个低质量资源。根据参与者的说法,在线资源的质量因几个因素而降低:其内容过时、是另一个在线资源的复制品、缺乏示例或存在糟糕的代码示例、缺乏适当的结构、缺乏相关资源等。答案示例包括:“一些教程开始时很好,但结束时非常糟糕,并且跳过代码部分,这会让初学者或对该概念不熟悉的人感到困惑”,或“即使 95%的人有完全相同的问题,也拒绝或无法提供确切的解决方案”。

Hard to filter (62 participants): A frequent complaint in our set of results is that online resources are sometimes hard to filter or retrieve . Even after using a search engine and an appropriate query to obtain an initial set of filtered results, a participant still needs to go over several of them to decide which of those are in fact useful. Participants also describe that some useful resources are unavailable , not free of charge, or hard to skim . Examples of answers are: “Difficult to parse through a massive amount of resources for specific topics”, or “Information scattered around, sometimes hard to find what you look for”. This, along with the previous category, raises the need for researchers to develop and promote tools — such as the ones described in Section II-C — able to mine, process, and recommend relevant elements of online informal documentation.
难以筛选(62 名参与者):在我们的结果集中,一个常见的抱怨是在线资源有时难以筛选或检索。即使在使用搜索引擎和适当的查询来获得初步筛选结果后,参与者仍然需要浏览其中的几个,以决定哪些实际上是有用的。参与者还描述了一些有用的资源不可用、不免费或难以浏览。答案的例子包括:“很难从大量资源中解析出特定主题”,或“信息分散,有时很难找到你要找的东西”。这一点,加上前一类,提出了研究人员需要开发和推广工具的需求——例如在 Section II-C 中描述的那些——能够挖掘、处理和推荐在线非正式文档的相关元素。

Hard to understand (36 participants): Respondents also expressed difficulties in understanding the content of some online resources. The most frequent complaint in this category is their lack of necessary background . In other words, the online resource does not consider the required knowledge to understand it, and does not take into account the background of its target audience. Additionally, respondents mention lack of examples , content that is too long or fragmented , the last one being defined as content that is spread across several resources, as other elements that make online content hard to understand. Finally, some participants report that online resources whose format does not match their learning preferences are also difficult to comprehend. For example: “Assuming you already know something. If a resource assumes you know something it should be on a list of assumptions (…)”. This suggests the need for tools able to properly compose (and give appropriate context for) elements of mined (informal) online documentation.
难以理解(36 名参与者):受访者还表示难以理解某些在线资源的内容。此类别中最常见的抱怨是缺乏必要的背景知识。换句话说,在线资源没有考虑到理解它所需的知识,也没有考虑到其目标受众的背景。此外,受访者提到缺乏示例、内容过长或过于零散,最后一个被定义为内容分散在多个资源中,是使在线内容难以理解的其他因素。最后,一些参与者报告说,格式不符合他们学习偏好的在线资源也难以理解。例如:“假设你已经知道一些东西。如果一个资源假设你知道一些东西,它应该在假设列表中……”。这表明需要能够正确组合(并为)挖掘的(非正式)在线文档元素提供适当上下文的工具。

Issues with videos (9 participants): Some respondents pointed out some of the things they do not like about using video for learning purposes. Among these issues, we find that videos are too long , often outdated , too shallow or do not have the expected level of detail in their content; they have a linear structure and can become repetitive . For example: “Most videos are far too long for something that could be covered in 10 or so minutes”. This confirms the need for tools such as the one by Ponzanelli et al. [49] .
视频问题(9 名参与者):一些受访者指出了他们在使用视频进行学习时不喜欢的一些事情。在这些问题中,我们发现视频太长,通常过时,内容太浅或没有预期的细节水平;它们具有线性结构,可能会变得重复。例如:“大多数视频太长了,可以在大约 10 分钟内涵盖的内容却要花费更长时间”。这证实了需要像 Ponzanelli 等人所提供的工具 [49]

Missing information (7 participants): Some of our participants complained about some desired information that is either missing or very hard to locate in online resources. Information such as best practices , theory , soft-skills , or data related to commercial products is referred as missing. Examples: “Some combinations of technology and solutions aren’t covered”, or “(they) don’t teach commercial, or enterprise technology or uses”. This indicates a prevalence of information for open source technology.
缺失信息(7 名参与者):我们的一些参与者抱怨某些所需信息要么缺失,要么在在线资源中很难找到。缺失的信息包括最佳实践、理论、软技能或与商业产品相关的数据。示例:“某些技术和解决方案的组合没有被覆盖”,或“(他们)不教授商业或企业技术或用途”。这表明开源技术的信息占据主导地位。

Bad collateral effects (3 participants): Some participants describe “mansplaining” ( i.e. , condescending answers) and hostile communities as phenomena they dislike in online resources. Example: “Communication-based resources ( e.g. , chatrooms, forums, Q&A sites, etc; as opposed to books and documentation) can often be hostile and it is always very hard to provide information at the level of expertise that will most benefit the receiver”.
不良的附带效应(3 名参与者):一些参与者描述了他们在在线资源中不喜欢的现象,如“男性说教”(即居高临下的回答)和敌对的社区。例子:“基于交流的资源(例如聊天室、论坛、问答网站等;与书籍和文档相对)往往是敌对的,并且总是很难提供最能使接收者受益的专业水平的信息。”

Other comments (28 participants): There are a few additional elements respondents included as dislikes. One is about a perceived absence of online resources directed to users with different levels of expertise in a topic. In particular, participants complained about the excess of resources for beginners compared with the number of resources aimed at intermediate or advanced levels. Additionally, participants mentioned bias , or attempts to evangelize instead of educating as an annoying element of some of them. Example responses are: “There is good content for beginners, good content for experts, and nearly nothing in between”, or ‘evangelisation (sic) and education: I want to be told how to use the tech, then I can decide if it’s good or not”.
其他评论(28 名参与者):受访者还提到了一些其他不喜欢的元素。其中之一是缺乏针对不同专业水平用户的在线资源。特别是,参与者抱怨初学者资源过多,而针对中级或高级水平的资源数量较少。此外,参与者提到了一些资源存在偏见,或试图传教而不是教育,这让人感到厌烦。示例回应包括:“有适合初学者的好内容,有适合专家的好内容,但几乎没有适合中间水平的内容”,或者“传教和教育:我想知道如何使用这项技术,然后我可以决定它是否好”。

3) Wishes: given the pros and cons of learning from online resources, participants expressed wishes on how such resources could be improved.
3) 愿望:鉴于从在线资源学习的优缺点,参与者表达了对如何改进这些资源的愿望。

Better content quality (57 participants): In order to improve the quality of online resources, the study participants suggested ways to fix the aspects they dislike about them. The most common fix they propose is the addition of good code examples and real world examples , followed by the inclusion of information about the required background needed to understand the content. Also, to solve the problem with outdated resources, they propose to include some annotations in case the resource becomes outdated . Example answers are: “More detailed explanation for what each section of the example code does”, or “Better information versioning. A current guide may link to how-to’s for outdated versions and require forum posts or mailing lists to fill the gap. Overall just good documentation upkeep”.
更好的内容质量(57 名参与者):为了提高在线资源的质量,研究参与者提出了修复他们不喜欢的方面的方法。他们提出的最常见的修复方法是增加好的代码示例和现实世界的示例,其次是包含理解内容所需的背景信息。此外,为了解决资源过时的问题,他们建议在资源过时的情况下添加一些注释。示例答案包括:“更详细地解释示例代码的每个部分的作用”,或“更好的信息版本控制。当前的指南可能会链接到过时版本的操作指南,并需要论坛帖子或邮件列表来填补空白。总体而言,只是良好的文档维护”。

More types of content (56 participants): Our respondents pointed out the types of content they would like to see more often in online resources. The most recurrent petition is to be able to obtain guidance or direction from experts in the field. Particularly, respondents would like to either access content created by reputable , experienced programmers or to directly contact experts in case they are stuck with their own projects, even if this involves a monetary contribution. Moreover, respondents would like to find the references and related resources for the content presented by an online resource. Participants also would like to see more content about soft-skills for programming. Examples: “More references to source material authors used for learning themselves (…)”, or “Additional reading to expand our knowledge”.
更多类型的内容(56 名参与者):我们的受访者指出了他们希望在在线资源中更频繁看到的内容类型。最常见的请求是能够从该领域的专家那里获得指导或方向。特别是,受访者希望能够访问由有声望、有经验的程序员创建的内容,或者在他们自己的项目遇到困难时直接联系专家,即使这涉及到金钱上的贡献。此外,受访者希望找到在线资源所呈现内容的参考资料和相关资源。参与者还希望看到更多关于编程软技能的内容。示例:“更多参考资料,作者自己用于学习的来源材料……”,或“额外的阅读材料以扩展我们的知识”。


Fig. 7
 - 

Soft-skills for the workplace - number of answer by level of experience.
Fig. 7  图 7

Soft-skills for the workplace - number of answer by level of experience.
职场软技能 - 按经验水平划分的回答数量。

Better accessibility (27 participants): According to the collected answers, participants would like to have better ways to retrieve and filter online resources . Particularly, they wish to have additional ways to index, categorize, and discover online content. In addition, participants are also interested in new ways to assess the quality or trustworthiness of online resources by using, for example, the reputation of the author of the content. Examples are: “Better indexing, categorizing, and discovery within resources”, or “Credentials and experience of author. This is not to say I wouldn’t consider one with less experience imparting information”.
更好的可访问性(27 名参与者):根据收集的答案,参与者希望有更好的方法来检索和过滤在线资源。特别是,他们希望有更多的方法来索引、分类和发现在线内容。此外,参与者还对通过使用例如内容作者的声誉等新方法来评估在线资源的质量或可信度感兴趣。例子包括:“更好的资源内索引、分类和发现”,或“作者的资历和经验。这并不是说我不会考虑经验较少的人传递信息”。

Better format (27 participants): Respondents also suggest additional ways to improve the format used to display online content. The most frequent suggestion is the use of summaries to either quickly explain the content of a single online resource, or to summarize the content of several related online resources. Other suggestions are the use of more practical/interactive formats, the use of conversational language as an alternative to highly-technical jargon, the use of more diagrams and pictorial resources, and the adoption of non-linear structures to describe a large topic. For the latter, some participants proposed the use of tables of contents, quizzes, or assessment tools to be able to jump through the content. Finally, participants also recommended additional ways to improve video tutorials, by including a (video) index as well as a transcript . Examples include: “More interactive tutorials”, or “For online documentation, tutorials, courses, etc., I wish there were surveys or quizzes you could do to determine where you are and it could suggest parts for you to skip through or not take the typical linear order that doesn’t fit everyone”.
更好的格式(27 名参与者):受访者还建议了其他方法来改进用于显示在线内容的格式。最常见的建议是使用摘要来快速解释单个在线资源的内容,或总结几个相关在线资源的内容。其他建议包括使用更实用/互动的格式,使用对话语言作为高度技术术语的替代,使用更多的图表和图片资源,以及采用非线性结构来描述一个大的主题。对于后者,一些参与者建议使用目录、测验或评估工具来跳转内容。最后,参与者还推荐了改进视频教程的其他方法,包括添加(视频)索引以及文字记录。示例包括:“更多互动教程”,或“对于在线文档、教程、课程等,我希望有调查或测验可以做,以确定你的位置,并可以建议你跳过的部分,或者不采用不适合所有人的典型线性顺序”。

Summary of RQ 3 : While participants appreciated the large amount and diversity of resources available online, they felt such resources are difficult to search and browse. They suggest the need for recommender systems aimed to improve their access to online resources, including video tutorials and to improve the comprehension of code examples.
RQ 3 总结:虽然参与者对网上大量和多样化的资源表示赞赏,但他们认为这些资源难以搜索和浏览。他们建议需要推荐系统来改善他们对在线视频教程等在线资源的访问,并提高对代码示例的理解。

E. RQ 4 - Soft skills
E. RQ 4 - 软技能

Finally, we asked our participants about the soft-skills they would like to learn more about using online resources, considering two main categories: soft-skills mostly used in the workplace, and soft-skills related to self-care and happiness as a programmer.
最后,我们询问了参与者关于他们希望通过在线资源学习的软技能,主要考虑了两个类别:工作场所中常用的软技能,以及与程序员的自我照顾和幸福相关的软技能。

Soft-skills for the workplace. Fig. 7 introduces the soft-skills for the workplace that our participants most frequently included in their answers. The numbers without parenthesis express the participants (per experience level and in total) that selected each one of the answers, and the numbers in parenthesis present the proportion of participants in the corresponding experience level that selected the answer. For instance, the answer “How it is like to work in a company” was selected by 120 participants in total: 59 of them were Students and the remaining 61 were Professionals; the 59 Students account for 73.8% of all the Students that replied to the survey, whereas the 61 Professionals represent 49.2% of all the Professionals.
职场软技能。 Fig. 7 介绍了我们的参与者在回答中最常提到的职场软技能。没有括号的数字表示选择每个答案的参与者(按经验水平和总数),括号中的数字表示在相应经验水平中选择该答案的参与者比例。例如,答案“在公司工作的感觉”总共有 120 名参与者选择:其中 59 名是学生,剩下的 61 名是专业人士;59 名学生占所有回复调查的学生的 73.8%,而 61 名专业人士占所有专业人士的 49.2%。

The difference in proportions between Students and Professionals having interest in learning about working in a company is to be expected, as many Students may have never worked in industry before and therefore would like to know more about it.
学生和专业人士对了解在公司工作的兴趣比例差异是可以预料的,因为许多学生可能从未在行业中工作过,因此希望了解更多相关信息。

We found that “How it is like to work in a company” is the most interesting soft-skill for our participants. Note that, even if students might not possess this soft-skill yet, they realize it has a paramount importance. Also “How to achieve a good work/life balance” is considered as important (54.9% participants). Interestingly, there was only a small difference in the distribution of this topic among Students (51.2%) and Professionals (57.3%). However, a larger proportion of Students (56.2%) want to know more about how to ask for help in the workplace, compared to Professionals (36.3%). With respect to understanding and negotiating salary, we found that even Professionals still express interest in knowing more about salary (35.5%) and how to negotiate it (40.3%). Proportion test indicated a statistically significant difference here ( p -value=0.001) with, of course, less interest of professionals for internships, but more attention to work/life balance.
我们发现,“在公司工作是什么样的”是我们参与者最感兴趣的软技能。请注意,即使学生可能还不具备这种软技能,他们也意识到它具有极其重要的意义。同时,“如何实现良好的工作/生活平衡”也被认为很重要(54.9%的参与者)。有趣的是,这个话题在学生(51.2%)和专业人士(57.3%)之间的分布差异很小。然而,相较于专业人士(36.3%),更多的学生(56.2%)希望了解更多关于如何在工作场所寻求帮助的信息。关于理解和谈判薪资,我们发现即使是专业人士仍然对了解更多关于薪资(35.5%)以及如何谈判薪资(40.3%)表现出兴趣。比例检验表明这里存在统计学上的显著差异(p 值=0.001),当然,专业人士对实习的兴趣较少,但对工作/生活平衡的关注更多。


Fig. 8
 - 

Soft-skills for self-care and happiness - number of answer by level of experience.
Fig. 8  图 8

Soft-skills for self-care and happiness - number of answer by level of experience.
经验水平与软技能对自我护理和幸福的影响 - 回答数量。

Self-care and happiness. As shown in Fig. 8 , more than 50% of all participants express interest in soft-skills such as self-esteem, health, or strategies to cope with exhaustion. There is no statistically significant difference between students and professionals ( p -value=0.90).
自我关怀和幸福。如 Fig. 8 所示,超过 50%的参与者对自尊、健康或应对疲劳的策略等软技能表现出兴趣。学生和专业人士之间没有统计学上的显著差异(p 值=0.90)。

Summary for RQ 4 : The soft-skills perceived as most important are the knowledge of how is like to work in a company, and (especially for professionals) the work/life balance. There is less attention to internship, especially from professionals.
RQ 4 的总结:被认为最重要的软技能是了解在公司工作的情况,尤其是对于专业人士来说,工作/生活平衡。对实习的关注较少,尤其是来自专业人士的关注。

SECTION V.  第五节。

Threats to validity  有效性威胁

Threats to construct validity concern the relationship between theory and observation. Given this a survey, the reported answers represent a perceived reality rather than facts observed from the field (for which different kinds of studies would be desirable). In some cases, students might have more limited experience than professionals, which may influence their answers to some questions (e.g., concerning acquiring programming languages/technologies, the importance of knowing how it is like to work in a company, etc.). To ensure that the provided answers cover all possible options, even the ones we did not foresee, we always gave the freedom to participants to manually introduce answers that we did not readily provide. Finally, it is also possible that some participants misunderstood some questions and this could have influenced some of our results.
对构建效度的威胁涉及理论与观察之间的关系。鉴于这是一次调查,报告的答案代表了一种感知的现实,而不是从现场观察到的事实(对于这些事实,不同类型的研究会更为理想)。在某些情况下,学生的经验可能比专业人士更有限,这可能会影响他们对某些问题的回答(例如,关于获取编程语言/技术,了解在公司工作的重要性等)。为了确保提供的答案涵盖所有可能的选项,即使是我们未预见到的选项,我们总是给予参与者手动输入我们未提供的答案的自由。最后,也有可能一些参与者误解了一些问题,这可能影响了我们的一些结果。

Threats to internal validity concern factors internal to our study that might have influenced our results. As shown, we have statistically analyzed the provided answers by experience level, and reported differences in proportions where these were found. It is possible that subjectiveness and imprecisions could have been introduced in RQ 3 , where open coding was necessary. We mitigated such a threat by having multiple evaluators iterating over the categories.
对内部效度的威胁涉及我们研究内部可能影响结果的因素。如图所示,我们按经验水平对提供的答案进行了统计分析,并报告了发现的比例差异。在需要开放编码的 RQ 3 中,可能会引入主观性和不精确性。我们通过让多个评估者对类别进行迭代来减轻这种威胁。

Threats to external validity concern the generalizability of results. Although we obtained 205 answers to our survey, some of the participants have been recruited among students in a university department and others through the social media connections of one of the authors. While the information shown in Section IV-A indicates a good diversity in terms of gender, degree, and professional experience, further studies involving a different samples of students (e.g., students from one programming course, students with diverse programming backgrounds and experiences) as well as experienced practitioners are highly desired. Finally, we targeted all CS students instead of a specific curricula, as we aimed at achieving a larger generalizability (and a larger number of participants), rather than focusing on a specific, well-defined group of students.
对外部效度的威胁涉及结果的普遍性。尽管我们获得了 205 份调查问卷的回答,但部分参与者是从大学系的学生中招募的,另一些则是通过其中一位作者的社交媒体联系招募的。虽然 Section IV-A 中显示的信息在性别、学位和专业经验方面具有良好的多样性,但进一步的研究需要涉及不同样本的学生(例如,来自一个编程课程的学生、具有不同编程背景和经验的学生)以及有经验的从业者。最后,我们针对所有计算机科学学生,而不是特定的课程,因为我们旨在实现更大的普遍性(和更多的参与者),而不是专注于特定的、定义明确的学生群体。

SECTION VI.  第六节。

Conclusions and Future Work
结论和未来工作

This paper reports the results of a survey with 205 participants aimed at investigating how developers and computer science students use online resources to learn Computer Science/programming topics.
本文报告了一项针对 205 名参与者的调查结果,旨在调查开发人员和计算机科学学生如何使用在线资源学习计算机科学/编程主题。

The results of the survey suggest that, although online resources are very valuable for developers and students, there is a great potential to improve them. First, new online resources could be more efficient by exploiting several formats to explain the same programming concepts, taking into consideration the variety of preferred resource formats by developers and students. For instance, a resource that explains the usage of a debugger can greatly benefit from a screencast format which combines visual and audio cues, since its content involves a step-by-step procedure; these steps could be also summarized in a small table of contents to allow users to skip unnecessary content. In contrast, a resource that explains how to implement a small feature in source code may not receive the same benefit from an audio-video format, since navigating to the part that offers the actual solution will not be as straightforward as giving a glance to a text document. As an example, a recent contribution to support the creation of programming video tutorials augmented with supporting resources was proposed by Bao et al. [2] .
调查结果表明,尽管在线资源对开发人员和学生非常有价值,但仍有很大的改进潜力。首先,通过利用多种格式来解释相同的编程概念,新的在线资源可以更加高效,同时考虑到开发人员和学生对资源格式的不同偏好。例如,一个解释调试器使用的资源可以通过结合视觉和音频提示的屏幕录像格式大大受益,因为其内容涉及逐步操作;这些步骤也可以在一个小的目录表中总结,以便用户跳过不必要的内容。相比之下,一个解释如何在源代码中实现小功能的资源可能不会从音视频格式中获得同样的好处,因为导航到提供实际解决方案的部分不会像浏览文本文档那样直接。例如,Bao 等人提出了一项最近的贡献,支持创建增强了辅助资源的编程视频教程。 [2]

For existing resources, the potential lies in the creation and refinement of recommender systems that automatically make their content more accessible to users that adopt different learning resources. Previous work has already addressed some of the improvements participants in our survey would like to see in online resources, such as suggesting related resources [68] , short video fragments with relevant content [49] , code extracted from videos [76] , among others. However, our results suggest there is still room for further improvement.
对于现有资源,潜力在于创建和改进推荐系统,这些系统可以自动使其内容对采用不同学习资源的用户更易访问。之前的工作已经解决了我们调查中参与者希望在在线资源中看到的一些改进,例如建议相关资源 [68] ,包含相关内容的短视频片段 [49] ,从视频中提取的代码 [76] ,等等。然而,我们的结果表明,仍有进一步改进的空间。

Our future efforts will focus on context-sensitive learning. In other words, we would like to better understand how each type of online resource can effectively support learning different programming tasks, such as implementing new features, debugging, or refactoring. As a consequence, we foresee specific recommenders able to provide developers with relevant online resources that are particularly suited for their working context.
我们的未来努力将集中在上下文敏感的学习上。换句话说,我们希望更好地理解每种在线资源如何有效地支持不同编程任务的学习,例如实现新功能、调试或重构。因此,我们预见到特定的推荐系统能够为开发人员提供与其工作环境特别适合的相关在线资源。

References

1.
S. Bajracharya and C. Lopes, "Mining search topics from a code search engine usage log", Mining Software Repositories 2009. MSR’09. 6th IEEE International Working Conference on, pp. 111-120, 2009.
2.
L. Bao, Z. Xing, X. Xia and D. Lo, "Vt-revolution: Interactive programming video tutorial authoring and watching system", IEEE Transactions on Software Engineering, pp. 1-1, 2018.
3.
J. P. Biddix, C. J. Chung and H. W. Park, "Convenience or credibility? a study of college student online research behaviors", The Internet and Higher Education, vol. 14, no. 3, pp. 175-182, 2011.
4.
G. Bougie, J. Starke, M.-A. Storey and D. M. German, "Towards understanding twitter use in software engineering: preliminary findings ongoing challenges and future questions", Proceedings of the 2nd international workshop on Web 2.0 for software engineering, pp. 31-36, 2011.
5.
J. Brandt, P. J. Guo, J. Lewenstein, M. Dontcheva and S. R. Klemmer, "Two studies of opportunistic programming: Interleaving web foraging learning and writing code", Proceedings of the SIGCHI Conference on Human Factors in Computing Systems ser. CHI ’09, pp. 1589-1598, 2009, [online] Available: http://doi.acm.org/10.1145/1518701.1518944.
6.
J. Brandt, P. J. Guo, J. Lewenstein and S. R. Klemmer, "Opportunistic programming: How rapid ideation and prototyping occur in practice", Proceedings of the 4th international workshop on End-user software engineering, pp. 1-5, 2008.
7.
P. S. Dadzie, "Electronic resources: access and usage at ashesi university college", Campus-Wide Information Systems, vol. 22, no. 5, pp. 290-297, 2005.
8.
T. Daradoumis, R. Bassi, F. Xhafa and S. Caballé, "A review on massive e-leaming (MOOC) design delivery and assessment", 2013 Eighth International Conference on P2P Parallel Grid Cloud and Internet Computing, pp. 208-213, Oct 2013.
9.
R. Dave, "An exploratory study on measuring impact of internet on students’ academic life of sp university", EXCEL International Journal of Multidisciplinary Management Studies, vol. 2, no. 5, pp. 154-167, 2012.
10.
B. Dorn and M. Guzdial, "Learning on the job: characterizing the programming knowledge and learning strategies of web designers", Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 703-712, 2010.
11.
B. Dorn, A. Stankiewicz and C. Roggi, "Lost while searching: Difficulties in information seeking among end-user programmers", Proceedings of the 76th ASIS Annual Meeting: Beyond the Cloud: Rethinking Information Boundaries, pp. 21, 2013.
12.
E. Duala-Ekoko and M. P. Robillard, "Asking and answering questions about unfamiliar apis: An exploratory study", Software Engineering (ICSE) 2012 34th International Conference on, pp. 266-276, 2012.
13.
D. Dukić and J. Strišković, "Croatian university students’ use and perception of electronic resources", Library Information Science Research, vol. 37, no. 3, pp. 244-253, 2015.
14.
J. Fleiss, Statistical Methods for Rates and Proportions, Wiley, 1981.
15.
N. Fleming and D. Baume, "Learning styles again: Varking up the right tree!", Educational developments, vol. 7, no. 4, pp. 4, 2006.
16.
N. D. Fleming, "I’m different; not dumb. modes of presentation (vark) in the tertiary classroom", Research and development in higher education Proceedings of the 1995 Annual Conference of the Higher Education and Research Development Society of Australasia (HERDSA) HERDSA, vol. 18, pp. 308-313, 1995.
17.
L. Freund, E. G. Toms and J. Waterhouse, "Modeling the information behaviour of software engineers using a work-task framework", Proceedings of the American Society for Information Science and Technology, vol. 42, no. 1, 2005.
18.
R. E. Gallardo-Valencia and S. E. Sim, "What kinds of development problems can be solved by searching the web?: A field study", Proceedings of the 3rd International Workshop on Search-Driven Development: Users Infrastructure Tools and Evaluation, pp. 41-44, 2011.
19.
S. Gottipati, D. Lo and J. Jiang, "Finding relevant answers in software forums", Proceedings of the 2011 26th IEEE/ACM International Conference on Automated Software Engineering, pp. 323-332, 2011.
20.
J. R. Griffiths and P. Brophy, "Student searching behavior and the web: use of academic resources and google", 2005.
21.
A. Grzywaczewski and R. Iqbal, "Task-specific information retrieval systems for software engineers", Journal of Computer and System Sciences, vol. 78, no. 4, pp. 1204-1218, 2012.
22.
A. Grzywaczewski, R. Iqbal, A. James and J. Halloran, "Software developers’ information needs: Towards the development of intelligent recommender systems", iUBICOM, vol. 11, pp. 66-74, 2011.
23.
E. Guzman, R. Alkadhi and N. Seyff, "A needle in a haystack: What do twitter users say about software?", Requirements Engineering Conference (RE) 2016 IEEE 24th International, pp. 96-105, 2016.
24.
E. Guzman, M. Ibrahim and M. Glinz, "Mining twitter messages for software evolution", Proceedings of the 39th International Conference on Software Engineering Companion, pp. 283-284, 2017.
25.
M. R. Harrington, "Information literacy and research-intensive graduate students: Enhancing the role of research librarians", Behavioral Social Sciences Librarian, vol. 28, no. 4, pp. 179-201, 2009.
26.
B. Hartmann, M. Dhillon and M. K. Chan, "Hypersource: bridging the gap between source and code-related web sites", Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2207-2210, 2011.
27.
D. He, D. Wu, Z. Yue, A. Fu and K. Thien Vo, "Undergraduate students’ interaction with online information resources in their academic tasks: A comparative study" in Aslib Proceedings, Emerald Group Publishing Limited, vol. 64, no. 6, pp. 615-640, 2012.
28.
A. Head and M. Eisenberg, "How today’s college students use Wikipedia for course-related research", 2010.
29.
A. Hussain and A. M. Abalkhail, "Determinants of library use collections and services among the students of engineering: a case study of king saud university", Collection Building, vol. 32, no. 3, pp. 100-110, 2013.
30.
S. W. Ji, S. Michaels and D. Waterman, "Print vs. electronic readings in college courses: Cost-efficiency and perceived learning", The Internet and Higher Education, vol. 21, pp. 17-24, 2014.