Internet Corollary - the ISP Angle

As a sidenote to the earlier navel-gazing about the future internet that outrgrew its parent thought, how does one serve up that media? I got into a discussion on Reddit which involved just that. I did a fairly in-depth amount of analysis for a total amateur on the subject, which I think should be posted here lest I ever forget why it would be a super-bad idea for me to attempt to start my own ISP.

A local/municipal ISP is responsible for the "last mile" connections around town to the residences, businesses, and government buildings it serves. These connections converge on a local data center (basically a building with switching and routing equipment that supplies IP addressing to its client connections and converts whatever media used for the last mile to fiberoptic via a router with SFP cages populated with fiber adapters). From there, the fiber out line needs to connect to the "backbone," which is inevitably going to be a fat fiber pipe owned and operated by an intermediate provider or a top-tier operator company (Comcast is one of these AFAIK, so if you're doing this where they operate the backbone you might STILL end up beholden to them to a lesser degree). Your ISP pays the backbone operator a fee for some amount of bandwidth up to the limit enforced by the number of fiber strands your datacenter has connecting it to their nearest backbone splice closet/datacenter. As I understand it these fees are not trivial. From there your ISP is allocated one of the IP ranges reserved for these sorts of things and you finally have a connection to the World Wide Web.

It should be noted the above summary only handles the basic problem of connectivity - not how you bill clients, not how you optimize your connection and cache data (caching is going to be important in order to not get raped by that backbone rent agreement, which typically charges by data used from what I've gleaned).

Yes, we all know what switches and routers are, and we may even know how to properly architect and provision IT infrastructure to avoid stupid bottlenecks in the transition from last mile media to backbone fiber, but there are technical, financial, and strategic problems with setting up your own ISP that make it such a bitch to get into, few ever do:

TECHNICAL CONSIDERATIONS

1. Redundancy and Disaster Recovery - when you're an ISP, people depend on you, sometimes with their finances, sometimes (as in the case of hospital patients or law enforcement officials) with their lives. Significant periods of downtime during inevitable equipment failures practically require ISPs to use geographically distant backup facilities with separate power (not just UPS/generator; ideally all the way to the plant), backbone uplink, and staffing. If one goes down, the other one needs to automatically come up within milliseconds, or you will face an unholy amount of network traffic congestion that could well overwhelm your backup site, killing that too as a knock-on effect. Expensive network engineers and linesmen need to be kept on retainer to handle the damage at all levels, even when they're not necessarily doing anything for you for the same reason.
2. Quality of Service - Because of the exhorbitant fees and expenses associated with getting that sweet, succulent backbone data to your subscribers and a desire not to pass that directly on to the customer who has competing offers from others with bigger pockets than you do, you pretty much need to get clever about how you cache, prioritize, and otherwise shape traffic. Load balancers, carefully-tuned QoS rules, ethically-questionable caching decisions, and other solutions must be leveraged to get the most out of what you've paid for in order to deliver a cost-effective service. Indeed, AFAIK ISPs commonly rely on these technologies to such an extent that they don't actually buy enough bandwidth to the backbone to literally serve all their customers to the utmost all at once; if everyone came online at the same time and couldn't be QoS'd adequately, they wouldn't have the pipe to handle it. A delicate balancing act.

FINANCIAL CONSIDERATIONS

1. Comcast and AT&T gon' find a way to sue you.
2. The startup costs of an ISP are very high because of the physical infrastructure and fees. The smaller your market gets (e.g. a 3,000 person town), the less likely your consumer base will be able to cover your expenses. Outside investment is typically not thrilled about a complicated business with high initial expenses, moderate year-over-year maintenance costs, plenty of failure points, a long wait for ROI, significant potential for competition, and significant hurdles to expansion into larger markets.

STRATEGIC PROBLEMS

1. Marketing expenses are difficult to swallow when you work in a market where tried-and-true is generally preferred by consumers over new and cheap.
2. Choice of infrastructure can have profound implications for the businesses' ability to expand with difficult tradeoffs in startup costs and future-proofing. A company with millions sunk into gigabit internet provided last-mile via coaxial MOCA technologies has to dump all of that infrastructure if customer expectations for bandwidth rise above what that technology can provide - this is partly why even though their technology can do it, cable companies have been loathe to tee up to what marks the last possible limit of throughput on their last-mile infrastructure. New capital expenditures will slow growth unacceptably, so ISPs need to make sure their wires and boxes will stand the test of time right from the outset. Additionally, opportunities to expand into new market areas may be missed if your infrastructure isn't adaptable enough to integrate more distant or heterogeneous endpoints - neat ideas like wireless mesh networks quickly die as the network grows because QoS and latency costs scale faster than the size of the network, making expansion untenable.

It's not that I think it's impossible to do, but I know we don't have the necessary smarts and money to do it. If I were going to try, though, I'd try something like this:

• Find 3rd party backbone-served area like Northern Illinois and its iFiber.org non-profit backbone organization with a decent potential market already served by Comcast.
• Coordinate with cooperative local governments to allow me to lease tower space on local water towers and other municipally-owned tall shit geographically near the backbone - they will earn a cut for this. Possibly build towers where potential for serving a juicy slice of the market is high.
• Get funded by investment from local businesses and possibly some kind of profit-sharing organizational vehicle that essentially crowd-funds me via local investment from stakeholders. Grassroots work a must to build this.
• Build out a 4G or even 5G infrastructure to serve business and residential traffic with wireless last-mile connections - this will likely require licensing, fees, and other government shit to be able to do.
• Customer plans do not go above 30mbps down/10mbps up to start. This is going to actually be fast enough for most people.
• Sell (at as close to cost as is feasible) transparently SSL-bumping cache boxes to customers with built-in 4G hotspots, 2.4/5ghz wireless APs, and 1x gigabit ethernet (remember that future-proof your infrastructure thing?), preferably with no ongoing service fee involved. This helps deliver perceptually faster service to customers, simplifies their network, reduces bandwidth utilization, and does so all without ethical compromise, as the device resides on their home network and uses off-the-shelf, user-auditable software/firmware/hardware.
• Hopefully utilize already-extant 3rd-part datacenter facilities and staff in the area to do the uplink from last mile to backbone.
• Focus on residential service over business customers at first, to soften the impact of inevitable learning mistakes leaving a bad impression about my reliability on my potentially most-profitable customer type.