A test of time: Modeling the long-term success of crowd collaborations

Abstract The Internet has significantly expanded the potential for global collaboration, allowing millions of users to contribute to collective projects like Wikipedia. While some of these collaborative efforts see early success, their long-term success is key to creating lasting impact. Despite its importance, this dimension of success remains largely unexplored. Prior work has assessed the success of online collaborations, however, most approaches are time-agnostic, evaluating success without considering its longevity. Research on the factors that ensure the long-term preservation of high-quality standards in online collaboration is scarce. In this study, we address this gap. We propose a novel metric, `Sustainable Success,' which measures the ability of collaborative efforts to maintain their quality over time. Using the English Wikipedia as a case study, we introduce the SustainPedia dataset, which compiles data from 48.7K Wikipedia articles. All articles in SustainPedia have reached the highest indicators of quality provided by the English Wikipedia at some point in their lifecycle. A portion of them (7.5%) were later demoted from this high-quality status, which we make use of for the purpose of this research. SustainPedia includes each article's label and more than 300 explanatory features such as edit history, user experience, and team composition. Using this dataset, we develop machine learning models to predict the sustainable success of Wikipedia articles. Our analysis reveals important insights. For example, we find that the longer an article takes to be recognized as high-quality, the more likely it is to maintain that status over time (i.e., be sustainable). Additionally, user experience emerged as the most critical predictor of sustainability. Our model provides the opportunity of automatically flagging articles at risk of being unsustainable. It also supports actionable factors that are significantly associated with (un)sustainable Wikipedia articles.