SoylentNews History

One of the most influential sites on the early Internet was Slashdot, a technology news site originally run by hobbyists that's still active to this day. This is the story of how the Slashdot community fragmented due to corporate meddling, and the formation of SoylentNews, a similar site founded as a spiritual successor to the "old days" of Slashdot (an effort our own NCommander was heavily involved in).

For the raw logs of the interview that documented this story, see SoylentNews History Log

Slashdot

Founded in the early 90s by CmdrTaco and CowboyNeal, Slashdot was a cornerstone of the early internet. As one of the first ever technology news websites, it was highly sought after; in the time of the dotcom boom, large "internet landmarks" like these changed hands often. Eventually, after many years of changing hands, Dice Holdings (operators of the Dice.com job site, among other things) acquired Slashdot from its previous owner, Geeknet (operators of Thinkgeek). This was a routine move for the site, as it had changed hands many times before -- but by this point (around 2012), most of the original editors were long gone. It was a different era for the site.

Post Purchase

Dice were very involved in Slashdot's operation, implementing many minor changes to the operation of the platform. This unnerved many of the long-time residents of the site, but there was a major change that proved to be the straw that broke the camel's back for the "old guard" of Slashdot -- the implementation of Slashdot Beta.

Slashdot's old interface (left) compared to the Beta (right). Source

NCommander comments:

Like it was this horrid whitespace meme; it turned a very information dense site into a wordpress block...It was like if every page of the print New York Times was double spaced.

The mission at this point was clear: an alternative would need to be created.

SoylentNews (is people)

Thankfully, true to the hobbyist origins of the original Slashdot, the software used to run the site was once open source. Although it was abandoned over the many acquisitions of the site, there was still a languishing Git repository on SourceForge containing a much older version of the original Perl source code for the Slash platform than the version currently in use by Dice; although it would take a lot of work to get running again, there was hope that with enough eyes and a bit of elbow grease, the old Slashdot would live again.

Several community members (including NCommander) gathered in the #alphaslash Freenode IRC chat and started brainstorming. The idea of new Slash implementations was nothing new -- alternatives like Pipedot and Technocrat were around, with brand new PHP implementations of a similar platform. However, this wouldn't do for the (soon to be) SoylentNews team. They would be dragging the original Slash kicking and screaming onto the 2014 internet.

What is Slash, anyway?

Slash is the original codebase for Slashdot, dating all the way back to the original launch of the site. Written by hackers and hobbyists in an era when hackers and hobbyists writing entire web applications was nearly unheard of, Slash is a mess of ancient Perl and old conventions. This isn't to say it's not clever, to be clear. A product of a time when CPAN modules were installed directly to the system and Apache's mod_perl was in its infancy, Slash's developers had to be... resourceful, to say the least.

Everything in old Slash is a static page by the time it reaches the user. This was important in a time of limited bandwidth and processing power. Tricks such as keeping scorecards in Apache's own shared memory space for task synchronization, a custom Perl-based cron replacement, and more were employed in order to keep server costs low, and response times fast, all the way back in the early-to-mid 1990s.

Information was found by BlackCoffeeDrinker regarding the hardware in use by Slashdot at one point [1]:

Slashdot currently has 16 web servers all of which are running Red Hat 9. Two serve static content: javascript, images, and the front page for non logged-in users. Four serve the front page to logged in users. And the remaining ten handle comment pages. All web servers are Rackable 1U servers with 2 Xeon 2.66Ghz processors, 2GB of RAM, and 2x80GB IDE hard drives. The web servers all NFS mount the NFS server, which is a Rackable 2U with 2 Xeon 2.4Ghz processors, 2GB of RAM, and 4x36GB 15K RPM SCSI drives.

Besides the 16 web servers, we have 7 databases. They currently are all running CentOS 4. They breakdown as follows: 2 Dual Opteron 270's with 16GB RAM, 4x36GB 15K RPM SCSI Drives These are doing multiple-master replication, with one acting as Slashdot's single write-only DB, and the other acting as a reader.

2 Dual Opteron 270's with 8GB RAM, 4x36GB 15K RPM SCSI Drives These are Slashdot's reader DBs. Each derives data from a specific master database (listed above).

Lastly, we have 3 Quad P3 Xeon 700Mhz with 4GB RAM, 8x36GB 10K RPM SCSI Drives which are sort of our miscellaneous 'other' boxes. They are used to host our accesslog writer, an accesslog reader, and Slashdot's search database.

Due to its age, getting Slash running was no easy feat. Because of the quirks inherent to such an old and strange design, it relied on ancient versions of Apache and mod_perl, with strange configurations. Thankfully, someone was up to the task...

Getting Slash going

NCommander got to work. Spinning up a new Linode VPS, they installed MySQL and compiled their own patched Apache 1.3 and Perl installations. Reliance on these old versions of the software stack would certainly have to change to bring changes like proper SSL support and security; but for now, just getting it working was the first priority. CPAN as a whole was getting old and crufty at this point -- dealing with the ancient CPAN bundle that Slash required was a monumental task, but one that could be conquered.

Nothing's Perfect

Unfortunately, there were looming issues with the SoylentNews team. The official leader of the team, Barrabas, was known for mismanagement; seeking total control of the project ruffled some feathers among the SoylentNews staff. As one example, the SoylentNews staff at the time were entirely grouped in the aforementioned Freenode IRC chat. Barrabas mandated that SoylentNews operate its own IRC server for discussion of operations and management. This move unfortunately cost the project some of its team. At this point, NCommander was hosting SoylentNews out of pocket, on two Linode 2048 instances that they were paying for themselves. Disputes over reimbursement, and the rift among the Soylent staff resulting from Barrabas' increasingly erratic management decisions grew and grew, until he was forced out of the team. Disaster struck. Barrabas announced he would be selling the soylentnews.org domain unless he received $2000 US dollars.

Getting the site back

This move sent the remaining SoylentNews staff (now de facto led by NCommander) into a scramble. Initially, NCommander was willing to pay; however, they were currently unable to cut a check, having been travelling internationally.

Thankfully, Linode provides subdomains for web services hosted by its customers. This link was published on SoylentNews in March of 2014 as a fallback in case the soylentnews.org domain went offline. Thankfully, Barrabas' access to the Linode instances currently hosting the site could be revoked, preventing any loss of data.

Eventually, Matt Angel (resident venture capitalist of the project) purchased the domain for $3000 US dollars. SoylentNews was back in the hands of its staff.

Incorporation

With the domain now safely in the hands of the Soylent staff, and Barrabas out of the picture, the team could once again get to work. On the advice of Matt Angel, the next target for the site was incorporation; this would allow for certain legal protections against the type of events that nearly cost the site its home. In addition, it would serve to diffuse power somewhat. NCommander writes:

...as things were setup and if I were so inclined, I could have turned off SN just by deleting the Linode's, and there would be no recourse for my actions nor would it be trivial to re-establish the site. With a proper legal corporation taking charge of the site, it prevents any one individual from taking over in a dictatorship. It has never been the plan for SoylentNews to be a dictatorship and this represents the first step that allows us to move to community governance.

After some difficulty (it's rather tough to incorporate a company with no real physical assets), SoylentNews was incorporated on July 4th, 2014.

Several of the original SoylentNews staff members were based around the US Northeast. A celebration was in order, and a major meet and greet BBQ was held. Times were good, and the project was on a positive trajectory once again.

Post-incorporation

With outlook for the project good, many ideas were floated for new directions for the site.

Subscription-based Soylent

At this point, SoylentNews was setting out to be more than just a news aggregator. One proposed change was a subscription-based donation model that provided services such as hosted mail and a shell account. The pushback for these changes was great -- as one user going by the name Buck Feta wrote:

I'd rather you use ads than stratify the community. How about posting the budget for the site, so we know what kind of number we need to make every month.

Sentiments in the comments of that post were divided. Some expressed similar opinions to the comment above. Some saw it as no big deal. But the majority of responses were negative. The changes had precident; projects such as Debian and Canonical had shell account access for developers, and had for years. Regardless, the SoylentNews staff heard the community's concerns, and never implemented the subscription model.

High Hopes

The ambitions of the SoylentNews team didn't stop there. The internal infrastructure of the site was somewhat advanced, due in part to NCommander's dedication to the original Slashdot they knew and loved. They write:

Like, if I was going to make a remake of the original Slashdot, I was going to make it a pinnacle to the gods.

More than just an aggregator

Perhaps the largest post-incorporation ambition for SoylentNews was for it to be more than just a news aggregator -- the goal was for it to become a source of high-quality independent and free journalism, with freedom of discussion and a right to criticism and opinions of all kinds.

The original SoylentNews manifesto, still available on the site, lays out the original direction the project was headed. A TOR exit node was soon put up by the staff (which proved surprisingly popular), and after one year of operation, NCommander held a celebratory 24-hour Twitch stream playing Nethack to celebrate.

I hope to play as a 24-hour event, from 0:00 to 23:59 EST, though that might change depending on how I'm feeling. I plan to setup a Skype session so members of the staff can join in and perhaps field a question or two. If anyone wants to live update for me (watch the stream and keep notes on what I've done) or something similar, drop me a Private Message on either the SoylentNews IRC channel or on Freenode (I'll be in #nethack while I'm streaming).

Unfortunately, NCommander's health had begun to decline, so some of the ideas outlined in the manifesto would never come to pass. However, they had one more major annoyance to solve...

Rehash

This is not something you do lightly Like this is a fucking effort

At this point in time, SoylentNews was still running on that revived ancient version of Slash. This was a massive annoyance, due not only to the difficulties of dealing with such old software, but also the potential security concerns of running 8 year old server software in 2014. Something had to change.

Thus, Rehash was born. The goal? Moving from Apache and mod_perl 1.x to a more modern version (Apache 2.0). Much of the codebase was clean, but a fair amount of it relied on quirks and conventions abandoned in the major version change. This was going to be a lot of work.

The first thing NCommander did was rip out as much code as possible: for one thing, pages would no longer be served statically to the user. In 2015, processing power was already cheap, and it was decided that the team could afford the processing hit in exchange for a cleaner codebase.

However, even without the 40000+ lines that were torn out, there was much work to be done. The CPAN dependencies for the project were a mess; and it still required a custom build of Apache. A major goal for this phase of the project was reproducibility: nobody would have to go through the headache of getting Slash running again. After hours and weeks of patching, it finally ran. On June 1st, 2015, Rehash 15.05 was deployed, running on Apache 2, mod_perl 2, and Perl 5.20. This brought many major changes and upgrades, in a massive wave:

- Rewrote large amounts of the site to migrate to Apache 2, mod_perl 2, and perl 5.20.
    > This was a massive undertaking. I did a large part of the initial work, but paulej72, and TheMightyBuzzard did lots to help fix a lot of the lingering issues. Major props to Bytram for catching many of the bugs pre-release
- Nexus Support (finally).
    > Currently we have the Meta and Breaking News nexii, with the possibility of adding more in the future, such as a Freshmeat replacement.
      > Nexii can be filtered in the user control panel under the Homepage tab. At the moment, this functionality is hosed due to unexpected breakage, but should be functional within the next 24-48 hours
- IPv6 support - the AAAA record is live as we speak
- Themes can be attached to a nexus independent of the "primary theme" setting; user choice overrides this
- Squashed More UTF-8 Bugs
- Migration to MySQL Cluster (more on this below)
- Rewrote site search engine to use sphinx search and (in general) be more useful
- Long comments properly collaspe now
- Support for SSL by default (not live yet)
- Fault tolerance; the site no longer explodes into confetti if a database or webfrontend goes down unexpectedly; allows for much easier system maintenance as we can offline things without manual migration of services
- Improved editor functionality, including per-article note block
- Lots of small fixes everywhere, due to the extended development cycle

(Doing several massive changes at once like that never works, by the way. It worked this time though. Good job, NCommander!)