Reassessing Tidyverse: Insights from a Veteran R User's Shift to Base R
After eight years immersed in the Tidyverse, I've made the conscious decision to pivot back to Base R. This choice stems from my recent collaboration with the University of Texas, where I encountered notable compatibility issues within the Tidyverse’s frameworks, particularly linked to the 'httpuv' package, crucial for both Tidyverse and Shiny applications.
My journey with R started back in 2015, primarily with data.table. Although my initial experiences were challenging, the Tidyverse soon captured my interest, transforming my approach to data analysis. However, recent interactions on a few pull requests (PRs) in GitHub’s comments section have left me dissatisfied and questioning whether Tidyverse's philosophy truly aligns with my goals. I encountered comments suggesting a lack of understanding of software development and attempts to introduce AI methodologies, which felt dismissive and off-base.
Redefining Software Interactions
As I worked to adapt 'httpuv' to reduce its dependencies, I resorted to R’s C API. Despite its dated feeling reminiscent of early computer programming, I find it effective for reducing complexity. My own history with computers stretches back to 1995, where I learned basic commands in my kindergarten years. This background gives me confidence in tackling issues at a more fundamental level.
Utilizing R’s C API isn't just a walk down memory lane; it reflects a significant shift in my approach to coding. Many developers today focus on high-level abstractions that, while user-friendly, can mask underlying inefficiencies. Working directly with R's C API forces a more granular perspective. This methodology not only enhances control but also streamlines the coding process, aligning better with my programming philosophy.
One of the main concerns for me is the communication culture within the community. Despite my enduring commitment and contributions to R—co-maintaining the CRAN Web Technologies task view and engaging in various package developments—the feedback on my contributions often didn’t reflect a collegial environment. The lack of any response to my new PR (PR 438) was particularly discouraging, especially after dedicating weeks to ensure compatibility for users at UT.
Reflections on the Community
Having spent a significant portion of my professional life with R—participating in coding sprints, bug fixes, and package maintenance—I’ve come to appreciate the vibrant collective of users and developers. Yet, these negative exchanges have made me reevaluate how I engage. What once felt like a supportive community now seems marred by condescending interactions, particularly towards those eager to contribute constructively.
This shift in tone feels disappointing, especially for new developers who are just finding their footing. A community should be a safe space for learning and sharing knowledge. Yet, too often, it appears that gatekeeping takes priority over fostering growth. It raises the question of whether the Tidyverse, once celebrated for its accessibility, might be straying from its foundational values. People want to contribute, but they also want acknowledgment and respect for their efforts.
As I work on my PhD thesis focused on enhancing R for econometrics, the decision to remove Tidyverse from my laptop felt like a necessary step. I’ll still tackle common data manipulation tasks but will do so using Base R functions as I work to streamline my processes further. The elimination of dependencies on the Tidyverse not only simplifies my coding but also minimizes potential issues and fosters a deeper understanding of the language itself.
The Implications of This Shift
There’s something to be said for returning to the basics, especially in a field that can feel overwhelming with its plethora of packages and frameworks. If you’re working in this space, you might find yourself asking: Is the latest framework really worth the complexity it brings? For many, the answer may lie in a hybrid approach, picking and choosing what truly serves your needs. Simplifying workflows often leads to better performance and easier troubleshooting.
Looking ahead, I’m enthusiastic about sharing methods for common data cleaning tasks with Base R. While I might be a bit rusty with functions like apply() and mapply(), this transition is a chance to reconnect with R's core functionalities. It's a false comfort to think that more layers always equal better results. My experiences serve as a reminder of the importance of communication and support in tech communities, and I hope others can benefit from reconsidering their own toolsets in light of performance and efficiency.
Stay tuned as I report on this journey and share practical examples that showcase a minimalistic approach to data handling with Base R. Achieving clarity and precision in data tasks doesn’t always require frameworks, and sometimes it's worth stepping back to consider how we can simplify our workflows. Simplicity is where the elegance lies.
