Wednesday, July 5, 2017

Why Do Things Cost So Much?

A few months ago Slate Star codex wrote a post about cost disease. I figured I'd publish some notes about it in a couple of posts.

Why do the following cost so much today in the US?

  • Education

  • Healthcare

  • Public infrastructure

(Housing - not included since growth in prices seem more reasonable and easier to explain.)

Compare current US costs with:

  • Cost in other countries - US may have slightly higher GDP per capita, but not enough to justify spending 2x - 5x more than other developed countries. 

  • Cost in the past.

  • Bonus - compare with cost of goods and services in other sectors (where the markets appear to work better).

Two questions:

  • Where is the increased spending going?

  • Why is this happening (especially compared to other countries)

Where is the money going?

In general, the US doesn't appear to be getting higher quality results than other countries. So where is the extra spending going?

  • Administrator bloat

  • More people employed to do the same things (compliance, redundancy, etc.)

  • Increased salaries and benefits

  • More "features" offered, even if overall outcomes aren't improved

  • More people getting services

Why is this happening?

  • Powerful public sector unions cause increased costs:

    • Higher salaries and benefits

    • Extra employees

    • Enforce extra requirements, restrictions on automation, etc.

    • Leaves the question - how did they get more powerful in the US of all places?

  • Government system

    • More divided

    • Common law? (See UK which also has increased expenses)

    • Politics is influenced more by money and special interests?

    • (See The American Interest)

  • Increased regulations - This can explain some of the changes over time, but is the US more extreme than other countries?

  • Lawsuits - In the US, people sue more often and for money, which raises costs:

    • Increased fees to balance risks of lawsuits

    • Expensive "defensive" actions to prevent lawsuits

  • Salaries increased to compete with other higher paying jobs.

    • This increase in salaries is greater than the GDP per capita difference between the US and Europe.

    • See post on higher Actual Individual Consumption in the US.

  • Changed values

    • Have more money to spend so spend a greater proportion of it on health and education since they're important, and other things are cheap anyways. 

    • Believe more in helping everyone achieve the same goals. Overall results may not show great improvement, but spend a lot on people who would have received less in the past.

The US is supposed to be more capitalist than European countries so one would expect it to have greater competition and reduced union power. However public sector unions appear to have greater power in the US than they do in other countries. This seems to be partially connected to the US political system and perhaps also connected to certain aspects of US culture.

In a future post, I plan to look into the specific areas in more detail.


If  anyone has the time, it would be nice to examine the evidence more carefully:

  • Find existing research on this topic.

  • Compare different countries and US states to see how spending changes based on specific factors.

  • Find people with personal experience who can explain some the differences between the US and elsewhere, or between today and the past.



Wednesday, January 8, 2014

The Learneroo Blog

I haven’t been posting much here lately, but check out the Learneroo blog for posts on Education:

Monday, September 23, 2013

Requirements and Product Planning

I found a college paper of mine on Requirements and Product Planning, so I figured I’d publish it online. You can view it on Scribd here or embedded at the bottom. Here’s a full outline (lightly edited) that I wrote before writing the paper:

Requirements and Product Planning Outline

Short product design plans combined with prototypes and diagrams are usually more effective means for finding and communicating the plan of a product than detailed written requirements. 

  • Finding and communicating the right requirements are the most important part of a project. It is therefore crucial to find the right approach.

  • The earlier you mess up, the worse is it is. The early decisions decide the direction of everything. So if you make wrong choice and only discover later, you will have to throw away a large amount of work. Studies showing how crucial the requirements stage is.

  • Definition of requirement – what the product must do to solve a user’s need.
    A full requirement answers these questions (Motorola):

    • Who it is for (E.g. The world, doctors, 54-year olds)

    • What it will do (E.g. Actions, results..)

    • Describe the full user experience and accurately represent the behavior of the software

    • Additional questions – Where, When and How well.

  • The purpose of requirements:

    • To accurately describe user experience and software behavior so all parties can understand it.

      • The requirements should be able to be communicated to customers, engineers, marketing and any other parties so they all know what the product will be.

    • To aid in finding the right product and refine until satisfactory

      • Sometimes will want to adjust the product, so need to specify first and then refine the requirements before building the actual product.

  • Ways to find ideas and determine the requirements

    • Market research tools and their limitations

      • Customer surveys and interviews

      • Analytics, data mining and social analysis

      • Site visits and focus groups

      • Usability testing.

      • Competitive analysis

This research may help gather data, but it will not determine what features or product to build. To determine what to build need to go beyond that.

  • Ideas and testing them with users

    • Need to determine how technology will be used to solve certain problems and what the user experience will be.

    • To do this, need to recognize the problem and understand what potential technology can be used to solve it.

    • This is how practically all successful tech companies built their products, not with market research. For example:

      • The Google founders invented a new algorithm to ranks search results which made them more relevant and higher-quality than what existed before.

      • Facebook developed a successful social network by figuring out what people would want most long-term in such a site. For example, they placed certain restrictions on users (like a real-name policy or set profile page) that helped them become successful.

They didn’t use market research, they were able to figure out what people would want and how to do it with technology.

  • Ideas are nice, but need to verify if they will work, since gain nothing from products that people do not want (at least in capitalist countries!)

As mentioned above, a primary purpose of requirements is to be able to refine product early in game to avoid large costs later.

  • Approaches to formulating and communicating requirements:

    • Full, formal text requirements (Motorola book)

      • This used to be the standard corporate practice in big companies.

      • These formal requirements are similar to an engineering process of e.g. a bridge or building.

      • In full, it can include five steps of engineering:

        • Elicitation – Discover requirements by consulting stakeholders.

        • Analysis and Negotiation – checks if requirements meet certain criteria, such as consistency, completeness and feasibility.

        • Documentation – for communicating them.

        • Validation – Check if it describes the system well and if there are problems.

        • Management – Manage the requirements. To track and develop complex requirements, can use a Database Requirements Management Tool, which provides various benefits. (Motorola)

    • Full prototype (Inspired book)

      • Create a throwaway prototype (using tools that make it fast) to show the various activities that will be possible. Parts that cannot be built into prototype should be simulated.

        • “Realistic representation of the proposed user experience”

        • Comprehensive coverage of the site – Prototype “should represent the functional requirements, the information architecture, the interaction design, and the visual design of the user experience.”

      • Another possibility is to use an evolutionary approach. Develop a very simple version of the site and then use that as basis for the actual site. However, if the foundations of the site are not well-designed, the site will always have problems.

    • Proposed middle approach

      • Short product design description, combined with wireframe/prototype to show overall idea and diagrams or visual representations (such as UML). Instead of full text requirements, can just use “Use Cases”.

  • Comparing the effectiveness of different approaches:

    • For discovering the right features and product, and testing them

    • For communicating to relevant parties (so they know what to do)

  • Each approach analyzed:

    • Full text requirements

      • Pros – Everything is spelled out. People should know what specifically needs to be done and when meet the criteria.
        Sometimes it is necessary to have very specific requirements – for example, if very specific critical criteria need to be met, such as software for a bank or nuclear plant.

      • Cons – Takes time to create, complex, but may be difficult to understand. Cannot test with users.
        The fundamental issue with full requirements is that they assume “It is possible to determine a stable set of requirements before system design and implementation starts.” With many web products nowadays, that is not exactly true.

    • Full prototype

      • Pros – Clearly shows how it works and this can be used to communicate well with all parties. Can also test product on actual users before building the real thing.

      • Cons – Can take a while to make and is redundant with later efforts. Additional problem is if it does too many details such as the UI, it will be describing “How” to do it before the “What” that requirements are supposed to be focused on.

    • Proposed best way – Compromise, but focused on the lean way – Minimum prototype, minimum product design plan (focused on use cases), clear diagrams, etc. to show all aspects of product.

      • Pros – Can efficiently create it, it can communicate effectively, can help find the right ideas, more flexible, allows for some testing before create product.

      • Why this approach is better than long detailed text requirements:

        • This approach fits better in web age as opposed to box shipped stuff. Since can iterate and develop new features quickly, do not have time for long requirement process.

        • Use cases and prototypes (and software tests) combined can meet the needs of requirements. The use cases describe the overall needs and the prototype can show the different parties what the product will do to meet them. Additional details can be shown with basic charts, but extensive detail is not necessary.

        • Outlines and visual charts are better since they communicate the ideas more quicker than long text. They can also be made more quickly, since one does not need to “textualize” the ideas by turning them into paragraphs.

      • However, this approach has some of the cons of each approach above, so need to find right balance. For example, can make sure prototype is built quickly with a focus on just communicating what the product will do.

  • While standard practice may have been to follow more formal and longer requirements system, this is becoming less common. Part of the reason for this is the overall differences between agile and traditional development methods:

    • Agile methods are adaptive rather than predictive.

    • Agile methods are people-oriented rather than process-oriented.

As mentioned, this is partially due to changing nature of web products where everything must move much faster. There is much less reason to use a complex formal process for developing products (especially consumer ones) on the web. Some companies may be used to older methods, or they may be developing products where it is important to get everything right. But this approach is no longer needed for most consumer software products.

View the full paper on Scribd

Saturday, September 21, 2013

JOBS Act and the Crowd-Sourced Startup

In April 2012, the JOBS Act was signed into law, but the SEC has been slow to implement it. The first part of the act will finally become active tomorrow, which will allow startups to publicly announce that they’re fundraising. However, they will still only be able to raise money from accredited investors, i.e. rich people.

Phase III of the JOBS Act will allow crowdfunding – ordinary people will be able to invest small amounts of their income in startups (see Crowdfunding passes in the Senate). This will help many small companies and startups raise money from a large number of people. Currently, a person or company can raise money on a site like Kickstarter, but can only offer backers rewards (like their product or tshirts), but not equity in the startup. Imagine how many more people will be interested in backing startups if they can hope to get rich from doing so! This will raise the risk of scams though, which is why there will be various regulations on crowdfunding once it is (eventually) implemented.

Startups will be able to raise money from the public, and could also use their “crowd” of investors to help to do things for their company. For example, a company could perform market research with their crowd investors, or ask them to help promote the company’s product on social media. Quirky uses its crowd to help decide what physical products to create, so tech companies could consult with their crowd to help decide on new features for an app.

Perhaps a company’s crowd could  be consulted with to work on a specific task, such as creating some icons for a site or improving its SEO. Crowd investors will want to be compensated for large tasks that they do, but it could still be easier to hire someone already invested in the company than an external consultant. In fact, maybe this work could be an alternative form of crowdfunding – instead of investing in a company, people could contribute work and get a share of equity. This would differ from standard employment for equity since it the work would be distributed to a large number of people. While this could make collaboration more difficult, many open-source projects have been successful with a large number of contributors, so perhaps startups can do the same thing.

Paul Graham once said:

I don’t think crowdfunding is good for startups. For startups, having large numbers of investors is bad, and having inexperienced investors is bad. So having a very large number of inexperienced investors is the worst scenario possible.


While too many ordinary investors could be a nuisance, a large group could by filtered through a crowdfunding site and can offer more value than standard rich investors. This may be why Paul Graham accepted the crowd-sourced startup FundersClub into Ycombiantor. Startups may even start crowdfunding because of the product and marketing opportunities it will provide.


Guide to Learning Programming

I’ve written about learning programming before and created a module on it on, but its also useful to have one post that helps you get started programming. I wrote such a post recently on Lifehack:
Guide to Learning Programming

Tuesday, June 18, 2013


I’ve written many posts on Zappable about education and technology, but I was always interested in doing more than just writing. I recently began working on such a project, one that will let people learn in a more interactive manner: – Learn by Doing

It’s still a very early version, but hopefully it will improve quickly! Since I’m now trying to practice much of what I preached, I copied some posts from here  to the Learneroo blog.

Wednesday, February 20, 2013

The Automated Store – Chart

Update: See the AutomatedStore

In previous posts, I wrote about the creation of an automated store machine in the 1900′s. In the chart below, I have jumped ahead to when the actual machine is first installed. I think this chart can help people get an overview of certain concepts by seeing them in physical form.  I created the rough sketch below with a pencil, but when I get better drawing software I will be able to create a digital version, which will be updated with color and more.
Machine Overview
While initially the main purpose of the machine was so the store owner could track inventory, many new features were added. This picture focuses on customers who which to view the inventory/catalog information themselves.
The customers enter their information by punching holes into a “HTTP” card they are viewing. This card then goes into the Router which cuts it up and sends it to the appropriate controller. For example, if a user asked to views product #43, the router will pass “43″ to the Product Controller’s viewing arm.
The controller will then send this information to the Active Record Player, which will send back the relevant data, as seen in the previous post. In this case, it would send back a copy of product card #43. The controller’s Action View will then take over and combine the data with the relevant templates to create a page. This will involve cutting up the data-card and inserting the information into the correct location on the “view product” catalog page. The page will then be combined with some general headers and some styling layers from the assets box, and then the whole thing will be pressed into a single page and sent back to the customer.