The Web’s Cruft Problem

cruft_header

The other day I came across this tweet from Kyle Simpson:

I don’t have a term for Kyle, but I completely agree with the sentiment, and I anecdotally get this sentiment from the greater web community. Between modals, app-install prompts, mobile web fails, ads, mobile redirects, EU cookie prompts, and the like, web developers—the people who collectively create the web—increasingly hate actually using the web.

In this article I’m going to argue that the reason for this hatred, and the biggest problem the web faces today is “cruft”: a term I mean to collectively refer to all the crap that the average web page includes that does not contribute towards what the user is actually trying to accomplish — read an article, buy a product, and so forth. I’ll also argue that the cruft problem is largely caused by a greater monetization problem that the web faces today.

To put this all in context let’s look at an example.

Cruft in action

The other day I saw this article on psychopaths appear on my Twitter stream. I opened up the article in Chrome for iOS and here’s what I saw:

cnn-web

This article makes for a good showcase of web cruft. All I wanted to do was read about psycopaths, as one does, but before reading I had to sift through a bunch of junk that I don’t care about—like social buttons, the temperature, and a terms-of-service modal — all for an article that’s about 2,000 words. I can’t even see the start of the article on my oversized iPhone 6+.

Loading this article took 200+ HTTP requests and used ~2MB of data. The article took about 3 seconds to load on my WiFi, and web page test says it would take about 13 seconds to load on an average mobile network.

I don’t bring up this example to single out CNN, because, as sad as this is to say, this article is now representative of the average web experience. According to the http archive, the average web page surpassed 2MB this May, and is now at 2.08MB. It’s not hard to find a far worse example out there.

Why the cruft?

I’m not the first to talk about the web’s cruft problem. Peter-Paul Koch recently described it as such:

“[W]eb versions of the articles have an extra layer of cruft attached to them, and that’s what makes the web slow to load. The speed problem is not inherent to the web; it’s a consequence of what passes for modern web development. Remove the cruft and we can compete again.”

And later he poses the question I’d like to tackle:

“The interesting question is: why all the cruft?”

PPK goes on to argue that the cruft is caused by tools, or more specifically libraries, frameworks, and the like that are increasingly used on the web. I agree with this to a point, as unnecessary tool usage can absolutely have adverse effects on page weight and loading times, but I believe there’s a more systemic problem at play here.

Let’s dive deeper into the CNN article. Among the 200+ HTTP requests the page makes are calls to 25 different domains.

web-inspector

Yes you read that correctly. TWENTY…FIVE. Among them are a few that are clearly ad related (ex. ad.doubleclick.net, pixel.moatads.com), a few that serve some analytics function, and many whose names are intentionally obfuscated to confuse us.

To me, you could phrase the “why all the cruft?” question a different way: since minimizing HTTP requests is one of the best known mobile web performance best practices, why do many mobile web sites flagrantly violate this rule?

You can certainly argue that part of the reason is tools, as using drop-in ads, social media widgets, and such will generate more HTTP requests than something hand-crafted. However I believe the answer has to do with money.

Follow the money

Why does CNN show ads? To make money. Why does CNN include tracking services? To learn more about the reader, to show more targeted ads, to make more money. Why does CNN use social media buttons? To get people to share the article, to get more page views, to get more ad views, to make more money. Why does CNN include a weather widget? Ok I don’t get that one; they should really get rid of that.

Again, I don’t mean to call out CNN as the “bad example,” but rather use them to show a specific example of a model that has become pervasive for content on the web.

My friend Brian Rinaldi recently wrote that the content model of the web is broken, in which he argues that we as web users thoroughly devalue content and writers. He argues that because we refuse to pay for content, content producers must resort to increasingly drastic tactics to make money off the content they produce — or have some ulterior motives to make the content production possible.

Paywalls have failed (mostly), so we’re left with a bunch of sites that use an eclectic set of ads, tracking scripts, modals, and such, all in an attempt to scrape together enough revenue to fund the content that lives behind the cruft.

What’s being done?

Many people are attacking the cruft problem, but interestingly the innovation is mostly coming from outside of the browser world.

Flipboard was perhaps the first big successful attempt at fixing cruft on the web. Flipboard essentially takes content from around the web, provides excerpts, and links off to the corresponding articles for the full content. This provides a rather nice browsing experience, where you don’t have to load a full article, and all the cruft that comes with it, just to get a quick preview of what the article looks like. For instance here’s a preview of a Fenway Park article shown in the Flipboard iOS app:

flipboard-article-example

But what’s more interesting is that Flipboard has gone beyond this content preview role, and now partners with certain content providers to display articles directly within the Flipboard app — foregoing the browser entirely. As an example, here’s the same psychopath article loaded in the Flipboard iOS app:

cnn-flipboard

Here I see the exact same content, although unlike the browser version, the Flipboard version of this this article is decidedly cruft free. Also unlike the browser version, this article loads nearly instantaneously, and a quick look at the network traffic shows why:

flipboard-ios-network

Whereas the browser version used 200+ HTTP requests from 25 domains, the Flipboard version uses 4 HTTP requests from 2 domains: cdn.flipboard.com and ad.flipboard.com.

As a company Flipboard has been successful, boasting 2014 numbers of 100 million+ active readers and a near billion dollar valuation, showing that there’s clearly a demand for a better means of reading content on mobile devices than what the browser currently provides. Flipboard’s success hasn’t gone unnoticed, and its business model has been more or less copied by a few others.

Flipboard Competitors

This May Facebook launched Instant Articles. Here’s how they introduce their service:

“As more people get their news on mobile devices, we want to make the experience faster and richer on Facebook. People share a lot of articles on Facebook, particularly on our mobile app. To date, however, these stories take an average of eight seconds to load, by far the slowest single content type on Facebook. Instant Articles makes the reading experience as much as ten times faster than standard mobile web articles.”

Sounds a lot like Flipboard doesn’t it? If you’re a publisher and you opt in, you let Facebook control the distribution of your content, in return for a far more performant experience for your readers, and presumably shared ad revenue of some sorts.

Let’s look at an example to see what this looks like in action. BuzzFeed participates in Instant Articles, and they recently published an article on 13 steps to instantly improve your day. As you might expect, the mobile web version of this article is laden with cruft, including two different plugs for their iOS app, social buttons, ads, and more:

buzzfeed-web

Like the CNN article we looked at earlier, BuzzFeed’s site is littered with ad scripts, tracking scripts, and the like. The final total comes out to 200+ HTTP requests and about 4MB worth of data:

buzzfeed-web-inspector

Let’s compare that to the same experience on Facebook’s Instant Articles, shown below:

instant-articles-example

As with with the previous Flipboard example, although the content is the same, this BuzzFeed article is remarkably cruft free. It also loads essentially instantaneously, and in my testing loading this article required just five network requests. (Facebook also appears to be employing some sort of prefetching algorithm to load parts or all of articles before you click on them, as the loading really does feel “instant.”)

facebook-network

There’s an argument to be had over whether this is a good business model for publishers, and it’s one that’s probably taking place in numerous conference rooms right now. But it’s hard to deny that Flipboard and Instant Articles provide a really elegant reading experience on mobile devices — something that mobile browsers have struggled with.

Flipboard and Facebook aren’t the only players in this game, as perhaps the biggest player in the tech world, Apple, announced that they’re entering this space with Apple News: a news app that uses essentially the same business model as Flipboard and Instant Articles.

Is this the end of the web?

No. It’s important to remember that regardless of how good of a user experience Flipboard, Facebook, and Apple provide, they’re still proprietary solutions controlled by a single company. These companies control how the content is used, and how people are able to access it. If Apple wants to lock down their content to Apple-created devices, they can do that.

Plus these companies all require some level of partnership with content providers to appear within their ecosystem, meaning these apps will only ever have a tiny fraction of the content the web provides. The ease of publishing and sharing on the web gives it a massive advantage over these proprietary solutions.

That being said, it’s hard to argue that the browser provides a better reading experience than what Flipboard and Facebook Instant Articles provide in their native apps. John Gruber might have said it best, in an article written in response to Facebook’s Instant Articles:

“I’m intrigued by the emphasis on speed. Not only is native mobile code winning for app development, but with things like Instant Articles, native is making the browser-based web look like a relic even just for publishing articles. If I’m right about that, it might pose a problem even for my overwhelmingly-text work at Daring Fireball. Daring Fireball pages load fast, but the pages I link to often don’t. I worry that the inherent slowness of the web and ill-considered trend toward over-produced web design is going to start hurting traffic to DF.”

What’s being done?

There are a few developments that should help the current situation for the web.

HTTP/2

The recently published HTTP/2 specification offers to substantially decrease latency on the web serving compressed HTTP headers, and loading resources in parallel over a single TCP connection. Once implemented in browsers, HTTP/2 should substantially lower the loading times of sites that rely on a large number of HTTP requests, such as the ones shown in this article.

Mobile-friendly

Last year, Google announced that they’ll penalize sites that aren’t mobile-friendly in their search results, as well as display a little “mobile-friendly” text next to sites that meet their guidelines in their search results.

mobile-friendly

This is a small tweak, but one that incentivizes developers and publishers of content sites, whom are often highly dependent on search-engine traffic, to keep their cruft to a minimum. And early research shows that it seems to be having a noticeable impact on search results.

Proxy browsers

Opera Mini has long been successful acting as a proxy browser, caching resources on its servers to reduce the amount of data that needs to be sent to each individual device. Chrome for Android and iOS now includes a similar option, and although it’s off by default, it’s still an option users have to help speed up the web.

Ad blockers

For many, ad blockers are the primary tool for attacking the web’s cruft on desktop devices, but they have yet to make their way into users mobile workflow, largely because mobile OS vendors have actively prevented them.

Google has a pretty good reason to actively discourage ad blockers, as they derive something like 80%–90% of their revenue from online advertising. Google made news back in 2013 for removing AdBlock Plus from Google Play.

Historically Apple has also prevented ad blockers, but that’s about to change. Apple recently made news by announcing that they’ll be opening an ad-filtering API to be used in Safari as of iOS 9. Regardless of whether this is a potential attack on Google, or a goodwill gesture towards Safari users, the end effect is reduced cruft for iOS users that opt-in to ad-blocking apps.

The future

Despite these various cruft-reducing features, I still believe this is an area ripe for innovation in the browser space. Why is it that as a publisher of articles your only real monetization option is injecting bulky ads that produce a worse experience for everyone?

I don’t have an answer here, and far smarter people than me have spent years trying to solve this problem. But still, it seems crazy that this is the best we can do. I still believe in the open web, and it’s not going anywhere anytime soon, but we really need to start thinking of ways we can start to clean this mess up.

Header image courtesy of Steven Depolo

Comments

  • Pingback: Dew Drop – July 7, 2015 (#2048) | Morning Dew()

  • Christian Heilmann

    Good points, but the final conclusion that taking openly available, accessible content and shoving it into closed app gardens is a slippery slope. All Flipboard and Facebook does is use the content and display it in a simpler format. We had this since 1997, it is called RSS. I can for example read the article of CNN just fine using feedly and subscribing to their feed. We shouldn’t replace cruft with subscriptions to third party sites or apps. That’s not using the web for its benefits. As terrible as the cruft experience is – I can still translate the article, get it read out to me, bookmark it for later or share the link with a friend. I can not do that with articles in apps. We shouldn’t let one bad practice lead to closed, better experiences. For example as a European or non-iOS user, I don’t get Facebook Instant Articles. That’s elitism.

    • I don’t remember concluding that taking confining content was a good thing. What I did conclude was that, for better or worse, these walled-gardens provide a monetization alternative for publishers—a monetization option that isn’t reliant on the bloated ads that often substantially degrade the user’s reading experience.

      Personally I’m not worried about a slippery slope in the sense that Flipboard or Facebook would gradually take market share from the web. As you point out, the walled-garden nature of these services actively work against that. What I do worry is about users’ perceptions of the web. I worry that if users see articles load fast in Facebook and Flipboard, but slow on the web, they’ll see the web as increasingly antiquated and slow. The web is capable of providing just as good of an experience, but it’s plagued with a bloated, often broken advertising model.

      • Flattr and other services have tried to create micropayments for online articles. These micropayment systems don’t seem to have worked either, but I keep feeling that’s the right _kind_ of solution to the monetization problem. I really wish a swath of consumers of web content would adopt some micropayment system.

        • Offer me a micropayment system that is (1) decidedly prepaid rather than postpaid and (2) gives me instant access to my account balance at any time and I’ll dive right in. Make it a nanopayment system and I’ll buy tons of content. I’m thinking Tracfone.com, but for content.

          • Sounds like a great startup idea. I would definitely be a (nano-)customer.

    • I was just about to reply with exactly the same thing. We all moved away from RSS to one-off custom, native mobile applications that need to be updated all the time and don’t have working links or basic features like text zooming, and social-ad-widget-explosion-sites because… because.. because we are a horrible race of monsters?

      • Because RSS is free. The copper bosses killed you, Aaron.

    • Frank Drebin

      Also, if you already see how cavalier most sites are with your privacy in the controlled environment of a browser, imagine how much apps can spy on you.

  • Richard Nash

    I think a big issue with this topic is the perceived value of an article in the mind of the user. Writing that is more thoughtful, composed, or considered educational is often perceived as more valuable, so education sites (udacity.com, teamtreehouse.com, etc…) seem to be able to make money off of subscription paywalls, where news sites don’t seem to be able to convince people that their content is worth paying for nearly as well. As a content creator this is tough, but as a user I often feel that the articles I read are not worth much. And sometimes the quality of some articles from a content creator affect my opinions of the other articles from that creator. So a listicle next to a serious piece make the the serious peice less valuable to me, especially when considered in aggregate. Tough issue, indeed.

    • As a user (an audience) I often feel that finding articles that are worth any time at all is like finding a needle in a haystack. I pay very little to anything web-based outside the amateur part of the blogosphere, by which I don’t mean content farms such as medium.com.

  • A single article on CNN is ~2MB to download via mobile device and Google still displays “mobile-friendly” for it on its search results page?? Confusing…

    • burkeholland

      Maybe “mobile friendly” means “looks ok on mobile”. They should just add the disclaimer “if and when it loads”

      • JeepinJason

        That’s pretty much exactly what it means.

    • JeepinJason

      The “mobile-friendly” tag has to do with the visual presentation of the site/page, ie: is the viewport configured properly, is it using proper font sizes for mobile viewing, etc. Basically, as long as you have either a responsive design or a “mobile” theme/design, it’s considered “mobile-friendly”.

  • James Morris

    I believe there’s so much cruft because there’s a lack of focus on the user a too much focus on the stakeholders. Departments wanted to stake a claim on real estate and will push this down to the people on the front line. This is often made worse, I believe, in places with in house teams, as companies will often only listen to outside consultants and ignore advice from inside.

    Another major factor is advertising. Many websites are free but to be free they’ve given up their user’s ‘soles’ to some extent, to make it viable. It’s a cache 22 really. As developers, designers, companies and entrepreneurs we need money to make things viable on the web but the perceived value of content is low so we’re left scratching around for ways to make money. Advertising the only real choice. But some good thinking and dropping much of the other clutter could make for a happier experience.

  • Jeff Edsell

    The part of web cruft I hate most:

    “Why does our page load so slow? Should you compress the images more?”

    Me: “No, it’s the five different tracking codes and the multiple social media buttons.” (Shows evidence)

    “Yeah, but we have to have all of those. So can you compress the images more? Maybe write less CSS?”

    • That’s a tough a conversation to have. And that’s part of what separates a good professional versus someone who is just going to roll over and do whatever the client wants.

      Unfortunately the other end of it is there’s always some agency or developer who will gladly take the client’s money and do whatever the client says, no matter the outcome. Again, follow the money.

      We want to make the client happy and sometimes putting an extra tracking code in there doesn’t take very long and we roll over in do it but it ends up hurting the web little by little. All of those little things can add up. Clients need to realize who is keeping their lights on… and it isn’t by listening to their CEO.

      • I’ve had this same discussion many times at my day job. The problem is that our sales team relies heavily on tracking data as part of the sales funnel. The point of our website is to bring in business, so it’s hard to argue that we should reduce tracking scripts and marketing pixels to increase conversions — the tracking scripts are the conversions we’re looking for.

        • Frank Drebin

          I think the real pest here is that everything has to be “cloud based”, so you can’t collect and analyze the data in-house. Everything has to be off site, and not every service provides everything you need so that’s how you end up with all that tracking code. Of course a company that doesn’t value the confidentiality of their own business data cares even less about their users/customers privacy.

    • my eyes are hungry

      My favorite. “Ok we’ve gotten it down to loading only 1 JS tracking file as a compromise.” (that file then spawns off 10 more HTTP requests for more tracking and DOM manipulation)

  • burkeholland

    My biggest problem, and I assume the problem that others have, is that optimizing for the web is hard. Because it’s so hard, folks like to assume that everyone has a fast connection because 2015. As a resident of a rural part of the US, my speeds range from 2 – 3 mbps down, and that’s on a good day. There are no real internet providers where I live, so most people consume content on their mobile devices over a mobile network. I would like to think that a majority of the world lives like this, but I’ve got no numbers to back it up.

    I imagine some will argue that it’s NOT hard, but I would present exhibit A – Paul Irish analyzing m.reddit.com. Just read through that issue. There are some items in there (optimizing images, ect) that are low hanging, but much of it is digging into trees inside the profiler and creating disc charts of the built bundles. I’ve been doing web development for over 10 years and I feel woefully inadequate to do what Paul did in that issue.

  • Pingback: The Web’s Cruft Problem | Web Development...()

  • serge stdrog

    this page makes 95 requests itself. Lol. That’s a lot of cruft

    • DarkLinkXXXX

      I counted 100. The files sizes weren’t that bad though.

  • my eyes are hungry

    Dunno why mobile pages/sites have social buttons on mobile to begin with, it’s built into the browser already (atleast on iOS). If desktop browsers pushed harder in this direction and took the load off the sites to supply them, then it could be a good win.

    • Ishmael Smyrnow

      This is already available via browser extensions or bookmarklets.

      • my eyes are hungry

        Everything is available with plugins/upgrades/patches. Face it, the average user is going with core and never changing anything. Hence the longevity of IE (“because it’s already installed on my computer”).

  • I have nothing to add to this conversation. TJ, you nailed it brother. Right on.

  • Pingback: Today’s Readings | Aaron T. Grogg()

  • Personal experience on “cruft”: many apps I have worked on ended up with extra ads or social links because somebody in the organization had the title of “Social Media Manager” or “Ad Manager” They were locally optimizing their career by creating projects that benefited their title but not the orgs overall goals. Ultimately, this was an organizational problem because these different roles were not organized/incentivized around a clear common strategy.

  • dehuszar

    The main issue at hand here is not a technical one, but a cultural and philosophical one. How should we fund the web? Proprietary solutions will always have a leg up in these scenarios because they are selling a license for a tool or service. The income can offset the content costs that would normally be paid for with advertising.

    Because we went straight into advertising as the monetary underpinning of web content, content producers have little recourse but to follow suit and project ads. It’s really not any different than the annoyance of having your favorite TV show interrupted while Company X tries to give you a pitch to buy a car, or laundry detergent.

    There’s a really good book that gets into the topic in more detail than I could here called The People’s Platform by Astra Taylor.

    Honestly, I think the only real way to do this is to create a public institution for the creation of content. Like a BBC for the web, but instead of the institution being the content creators themselves, it would be a way of compensating creators of popular, quality content as a public good.

  • David Piepgrass

    Ahh Disqus – comment cruft. Anyway, I disagree with the assertion “we refuse to pay for content”. No, users refuse standard paywall models like “$9.99 per month”. I’ve always thought that a one-click micropayment model might work, especially if coupled with a monthly ceiling / day pass. When’s the last time you saw “Skip the ads for 1¢! 24¢ to be ad-free for 24 hours!”? I’m disappointed that the industry hasn’t even begun to solve the one-click micropayment problem.

    • Frank Drebin

      It’s always easy to blame the customer, but that’s the kind of attitude that will kill your business. Since I don’t need to buy a single news paper and instead get to read the best of every newspaper out there I don’t need the same thing regurgitated, copied, rephrased in every place – I can get the same news anywhere. But instead of focusing on things to stand out like research, fact-checking, maybe even proofreading they throw the last bit of journalism over board and make underpaid interns blog their opinions about anything. And that’s when you start your news diet…

      Also, if I have to pull 2-4MB over a mobile connection I have already paid my ISP a bunch for the mobile data.

      I like the micropayment thing and have used it in the past (flattr), but that doesn’t really work for the large news sites.

      • David Piepgrass

        Huh? I didn’t blame the customer. I’m saying that there’s a business model that might work, but cannot be attempted because a frictionless micropayment system does not exist. Flattr is not what I had in mind, although I see how some folks (not me) might prefer the fixed-cost-donation model.

        • Frank Drebin

          I’ll edit my comment, I was a bit unclear. I meant to say that often times content producers blame the customer.

      • Frank Drebin

        There is a service now in Germany (and EU I guess) called Laterpay, before reading you can agree to pay a certain sum at a later date, once it reaches more than EUR 5 the money is paid out. I don’t know if you can opt out of paying if the content turns out to be crap though.

  • lmm

    Does Flipboard help CNN monetize? If so, why and how? Whatever it is that allows the Flipboard version of that CNN article to (presumably) make just as much money while sending 4 requests rather than 25, why can’t the exact same magic be applied to the web version?

  • swagv

    But it’s not just the Web. CNN’s mobile app is one of the biggest jokes out there. If I hired an 8-year-old who wanted to create a cheap native mobile app with lots of inane video links, the kid would largely come up with something that is 90% accurate to the CNN mobile app. So organizational culture also plays a role here.

  • Doug Smith

    As a web user I’ve learned to avoid cruft sites. I get all of my news on hacker news (like this article). I check news.google.com which has no cruft. I also use Flipboard mostly on Android because web browsing on Mobile has become painful. When I click an article and the cruft starts popping up I close the page or look for a less crufty site to find news. This is not a thought out behavior. It’s a subconscious behavior that evolved as websites get more crufty. I just inherently know when my time is being wasted on cruft and avoid it. CNN is actually a great example because 15 years ago that was the site I went to, to read news.

    My point in posting this is to bring up a thought. Many sites are probably driving away people without realizing just how many people they drive away. I guess if someone is paying for ads it doesn’t matter to them. At the end of the day though, I wonder, how effective can all that ad publishing really be.

    • Oh yeah, I have a clickbait domain list at least a hundred long stored in my head. What’s strange is that I invested zero effort in developing it, and it’s 100% accurate. Nothing like pain to train the brain.

  • the_robin

    Interesting that you work for Telerik, but you use Charles instead of Fiddler…

  • SAllier

    That is why frameworks like Rails include Asset Pipelines and Uglifiers to lower the connection count and the size.

  • Filip

    Ironically, I have to use “reader mode” to read this article, because it uses custom fonts, and Safari prefers to display a blank page to using the wrong font, so I was just staring at a white screen, even though all the text had loaded.

  • Great article! Just to reiterate some of what has been said here about cruft and its performance, it is simply a bi-product of not just ads and widgets but the trend we are seeing in development where everything is becoming a library or framework. I see now pages that could have been done with a little home grown JS and CSS being implemented with like 3 JS libraries and 2 frameworks. Developers are using 1% features of jQuery for instance and the other 99% is just bloat.

    Also as someone who helps a lot of developers I have to constantly ask people “Why are you bringing in jQuery and calling this function when you could have just written 2 lines of vanilla JS and do the same thing?”

    We are also bringing in these large frameworks that then try to streamline themselves. Here is an idea, try to not bring in the large framework and you won’t have to worry about streamlining it and its assets.

    I guess what I am trying to say is, less is more. Both in design but especially the code we develop. 🙂

    • People use jQuery because the design of Javascript (as contrasted with, say, Ruby with its assortment of zippers and collectors and the like) is notoriously poor in elegant ways to automate certain repetitive tasks. How often do people ask in coding forums, what’s the elegant way to so such-and-such to an array, only to be told that the short answer is that there isn’t. Then DOM takes that problem and compounds it by introducing object classes such as NodeList, which is like an array in that you can iterate over it with a for loop, and unlike an array in that you can’t do map and forEach and the like. Javascript without jQuery is a punishing experience.

  • andrea paiola

    Well… I try to do something… http://andreapaiola.name/magpie/

  • Pingback: The worsening web | Smash Company()

  • Ehsan Akhgari

    Is there something that we can change in browsers or browser engines to help with this problem? (I work on a browser engine 🙂

  • Proprietary app articles load faster because they are cruft free: for now. But what happens when these app publishers decide to ramp up the cruft to make more revenue?

  • Pingback: Bruce Lawson’s personal site  : Reading List()

  • codeWhisperer

    Perhaps the term Kyle was looking for is “hackenfreude”

  • One obvious question was not asked: if we accept that ads are necessary since we do not want to pay for the content, why do websites not have efficient trackers and ads? We do not need a lot of java and objects to track what a person likes and we do not need an overly complex advertisement system. And those sites that find it necessary to have a heatmap and all its overhead are ridiculous and make the user experience worse. The advertisement system of flipboard shows that you can have trackers and ads efficiently, so why don’t the websites have it?

  • Is there a term for that other than privilege? No, there is not. My perspective as a developer wannabee (github.com/n8chz) is that getting paid for doing something that requires intelligence is prima facie evidence of privilege.

  • Very thoughtful article TJ. I think one omission here is how cruft will affect the majority of the world’s population who have yet to log on to the Internet (about 60%: http://www.mckinsey.com/insights/high_tech_telecoms_internet/offline_and_falling_behind_barriers_to_internet_adoption).

    Sure, infrastructure for carriers will improve gradually, but as bandwidth increases in areas where web content producers reside, rural and developing nations will be at the mercy of slower connections for the foreseeable future.

  • Pingback: Michael Tsai - Blog - Safari Content Blocker and Web Advertising()

  • Pingback: Weekly Links #31 | meshedsociety.com()

  • The reasons for this grim situation are many, but they can be condensed to two main truths:

    1. The ad world is a cosmic horror with tendrils in nearly every universe.
    2. Maintaining technology costs money.

    Ads creeped their way onto the internet almost the moment it became a potential (and then realized) revenue stream. Ads follow the trail of profit. Always. On the internet, they acquired a trait that explains their ubiquity as a source of income: they are very passive. They don’t require much setup, and many of them depend on per-view or per-click mechanisms. Banner ads and such are pretty much set-and-forget.

    As you said, the incentive to follow the money and take steps to get more eyeballs (ugh) on a page is suffocating. And when social media opened the floodgates of connection, share icons started rutting like rabbits. And so the web accumulated cruft until people saw the profit potential in making apps that stripped the cruft. My God… we’ve come full circle.

    Joking aside, we have a few options. Not all of them are feasible. So I’ll list them in increasing order of success.

    1. The Google strategy taken a step further: penalize for the load added by ads and tracking.

    Why this won’t work:

    Two words: Google Ads. Perhaps hyperbolic to say, but if you run into an ad on the modern web, good chance it’s being served by Google. Think about the volume and reach of Google Ads, how many people depend on it, and you’ll see this is a non-starter. Even if Google optimizes Adsense to have almost no footprint in load time, that still doesn’t solve the original problem of cruft.

    Google has their fingers deep in that sweet advert pie and may not pull them out anytime soon. Penalizing sites that use ads would be a sign flashing: “Hey! Don’t use one of our profit channels! You’ll lose money and exposure!” Unless Google penalized for ads not served by their service, which would be a different problem entirely.

    2. As developers, we could enforce an ad and tracking plugin ban.

    Why this won’t work:

    Ad placement and creation, unfortunately, create a chain of responsibility that dips into our pockets. If professional developers stopped implementing ad and tracking plugins on principle… well, there’s the next rung of not-so-professionals. So the problem still wouldn’t be solved as not every dev will turn their back on money for principle. Not derogation, just fact. And people who depend on the internet for revenue won’t easily give up such a potent, no-headache-except-for-users stream of income.

    3. Let cruft-stripping apps take over.

    Why this won’t work:

    Sure, if you just want to bandage the problem while the internet festers underneath. And then the internet would die from neglect, and we want to save the web, yeah? And my earlier point about ads following profit means these apps wouldn’t be far off from the same problem once someone’s pupils turn into dollar signs. We’d just end up giving those “pay to get rid of ads” practices another arena to play. And those tend to be more intrusive to the experience. So that’s not really an option.

    4. Alternate revenue streams that are inconspicuous and based on quality content.

    Why this would work (if we could pull it off):

    Many have already suggested it, but something like Patreon that let visitors give back to sites that provide them value would leave clickbait and such without legs. The incentive chain would shift from pageviews to value. I bet if we set up a system where only good content (endorsed by people) gets rewarded, even CNN would fall over itself to kill the cruft.

    That said, I’m not placing blind faith in such a system because everything is exploitable. I can foresee a situation where the incentive for companies becomes trying to game you into clicking yet another icon. And that might lead to ads that shout about the great content and value a site provides. The net effect might just be another recruit in the social icon army and a putting a bunny bonnet on the ad beast. Of course, I may be looking through a haze of disillusionment.

    No matter what the solution ends up being, we’d have to hit them in the revenue. If advertising and traffic tracking ever represented a significant net loss, I bet content providers and companies would drop it like it caught fire.

  • Simon

    After loading and scrolling down, this article produced 120 requests.

  • Pingback: “I Should Get That” and FTB | For the Boys()

  • Pingback: What's Wrong with the Web? -Telerik Developer Network()

  • Pingback: Goodnight, Moon – TR Dispatch v2n11 | TechReckoning()

  • Pingback: Geek and Seek » What’s Wrong with the Web?()

  • Pingback: Wednesday is Link Day Still Breathing - online home of Chris Taylor - web developer, musician, geek()