Why Data Science Can't Find the Needle in the Haystack

A colleague of mine wants a predictive model. He is trying to determine which people on a health insurance plan will visit the hospital in the next couple of months. He has pretty good data for making this type of prediction. He knows who visited the hospital in the past; and he knows they are more likely to revisit. He knows what illnesses these people have, and which illnesses likely result in hospital visits. He knows what drugs have been prescribed, and whether patients are taking their drugs.

Even with this data and even with very good models, he still complains that the predictions are not good enough. His problem is too many false positives. And he simply doesn’t have enough employees to review every patient the model predicts.

Data science is a great tool, but it is not perfect. And if you are going to weld the data science tool, you should be aware its’ shortcomings. Data science is simply not very good at finding a needle in a haystack.

This concept can be illustrated with an example. Say a banker is managing the mortgages of 500,000 homeowners. He knows from experience that roughly 1,000 of these homeowners will default on their loan. There is plenty of data to help zero in on these 1,000 people: zip code, income, payment history, and credit rating. He knows that if I can put the right people some assistance, they may not default on their loan.

We have the data of build a predictive model. But regardless of how good the model is, it will not be perfect. When the model is run, each homeowner will be classified as at-risk for default or not at-risk. In this scenario, there are four possible outcomes for each homeowner. The homeowner is at-risk and is properly identified by the model; the homeowner is not at-risk is properly identified by the model. These are the two accurate predictions.

Every predictor gets some wrong too. When the homeowner is not at-risk, but the model says he is, that is a false positive. If the homeowner is at-risk and was not identified by the model, that is a false negative. In statistics, false positives are also referred to as “type I errors”. False negatives are referred to as “type II errors”.

Now let’s say that we construct a model that is 80% accurate, which is a rule-of-thumb threshold for a good prediction. With 80% accuracy on 500,000 loans, 400,00 will be correctly predicted and 100,000 will not. At an 80% prediction rate, 800 of the 1,000 homeowners would be predicted correctly.

Of course, an 80% success rate means there is also a 20% failure rate. I mentioned that 100,000 are not predicted correctly. There are 200 false negatives that are the balance of the 1,000 target loans. Subtracting the 200 false negatives from the 100,000-people identified incorrectly leaves 99,800 false positives. That is the extraordinary 500 times as many false positives as correctly predicted at-risk loans.

Even if the model can predict at the incredible rate of 99% the numbers of false-positives will outnumber the correctly identified at-risk cases by nearly five to one.

The problem here isn’t with data science or prediction methods. It simple math. When trying to use statistics to find a very small number among a very large number the false-positives will always greatly outnumber the actual positive prediction. This type of problem is truly a needle in a haystack.

Data Science isn't Rocket Science

Science is intimidating. It conjures up images of lab coats, telescopes and microscopes, or petri dishes and chemicals. But Data Science isn’t rocket science. Science is about discovery and Data Science is discovery in data.

You don’t need an army of PhDs for successful data science. Instead you need a basic understanding of statistics and a little computing power. Then follow this recipe of five steps to put Data Science to work for your business.

Step 1. Decide what to predict

First and foremost, you must know what you want to predict. Data Science in business is about making predictions. That predicting might be finding customers that will buy a product; or which patients will be readmitted to a hospital; or when to buy shares of stock.

Everyone knows the story of Target sending coupons for pre-natal items to a teenage girl, only to surprise her father. But that case didn’t happen by chance. Instead, someone at Target decided to specifically focus on pregnant women. That person decided to predict which of their customers were pregnant.

Make your prediction on a single thing. That thing will be represented by a prediction variable. The variable might simply be “Yes” or “No”, such as in the Target pregnancy example. It could be an item in a list, such as a day of the week. It frequently will be a number, such as the price of a barrel of crude oil.

Step 2. Make your hypothesis

Your hypothesis is the key to the puzzle. And yes, the word hypothesis comes right out of the scientific method because that’s what we’re doing; we’re applying the scientific method to data to make predictions. You decided in the previous step what to predict, now it’s time to guess how to make that prediction.

Some insight into your problem is helpful here. For example, in the case of Target above, someone had made the presumption that items a person purchased could indicate whether they are pregnant. They very likely narrowed the items down to a specific list of items or types of items. The hypothesis may have been as simple as “someone who buys pre-natal vitamins is likely to be pregnant”. Or even more specific, “a woman between 18 and 45 who buys pre-natal vitamins is likely to be pregnant”.

Step 3. Get the data

Of course, this whole exercise assumes the data is available to make these predictions. You will need to collect data points on any attribute you are testing, as well as the values you wish to predict.

Again, going back to our example to finding customers who are expecting. Assuming we want to build a model on the hypothesis that “a woman between 18 and 45 who buys pre-natal vitamins is likely to be pregnant”. We will need the following data:

• ·        A list of customers who bought products.
• ·         A list of the products they bought.
• ·         For each customer, we need their age and gender.
• ·         For the products, we need to know which are pre-natal vitamins.
• ·         And most importantly, we need to know who is pregnant.

In this age of big data, you may have all the data available. That is, every customer, and every product they purchased. If you have all the data, great, you can build your model based on the population, which is the statistician’s way of saying “all the data”. Otherwise we will use a sample, which is a way of saying some of the data.

Some data points may be difficult to obtain. In our example, the fact that a customer is expecting a child may not be readily available. In this case, special steps will be required to obtain the data. Target may have performed a customer survey or used some other means of gathering the information directly from the individual.

Step 4. Build a model

Building a model is where the fun starts. This is the statistical model, or predictive model. A basic understanding of statistics and knowledge of modeling software is necessary.

Before building a model, you should run some analysis to see if your hypothesis is worth pursuing. The typical first step is testing a null hypothesis for statistical significance. The null hypothesis checks that the predictor variable affects the prediction. In our case, the null hypothesis would be “knowing a person is a woman, between 18 and 45, who bought pre-natal vitamins has no impact on their being pregnant.”

In short, we compare the number of pregnancies in a random sample of customers against 18 to 45-year-old women buying pre-natal vitamins. If the difference in pregnancy rates between these two groups is greater than 5%, the null hypothesis is disproved, and our assumption is considered statistically significant.

Some caution is needed here, because a 5% difference could be attributed to improbable random samples. You can protect yourself against improbable results by repeating the test against additional random samples.

Once you have your data. And your hypothesis is sound. Building a predictive model is relatively simple. For example, the R code for creating a model looks like this:

modelFit = train(class ~ .,method="rf",
data=trainingCV, prox=TRUE)

And the code for making predictions looks like this:

prediction = predict(modelFit, testCV)

This code is illustrative to show that modelling does not require complicated commands.

Step 5. Check your results

Before creating your model, divide your data into two sets; a training set and a test set. The training data is used to create the model. The test set is used to demonstrate that it works. Typically, you would split the original data such that 80% of it is used for training with the remaining 20% used for testing.

In our sample, let’s say we have 1,000 customers in our data. We would randomly select 800 for training and 200 for testing. But there is nothing sacred about an 80/20 split. In fact, if your dataset is very large, say 100,000 or more, you could create multiple test sets using a 60/20/20 split.

When your data is prepared, create your model; if using R then run the train() function. Then take the created model and run it against the test data; if using R then run the predict() function. The predict() function will make prediction of whether the customer is pregnant or not.

The results of the prediction are compared against the original data to determine if our model makes reliable predictions. Using our sample, we would make predictions against 200 people. If the model correctly determines pregnancy in 150 of them, then our prediction was 75% accurate.

Your software should be able to provide you with an Area Under the Curve (AUC) analysis. AUC values close to 1 indicate a very good model, those near .5 are little better than flipping a coin. You should strive for AUCs of .80 or more.

And now we’re done

Well, maybe we’re not done. If you’re AUC is poor, then you should start the process over. But it’s not a complete loss, because even a bad model gives knowledge; knowing what doesn’t work is important too.

Of course this is a blog post. And I have over-simplified every step. Still, creating predictive model is not as intimidating as one might think. It’s most definitely not like putting a man on the moon.

What I am reading | "Ultra-Marathon Man"

Ultramarathon Man: Confessions of an All-Night Runner
Dean Karnazes



I finished this book recently. It's a pretty entertaining read. While Dean is obviously not a writer by trade, his stories draw you in. While I have no desire to run ultra-marathons, the stories inspire me to continue my running. I found the marathon to the South Pole to be particularly interesting. In any case, this is an easy read; I recommend it for runners at all levels.

What I am reading | "Don't Make Me Think"

Don't Make Me Think! A Common Sense Approach to Web Usability
Steve Krug

I'm slogging my way through this guide to designing easy to use web sites. I had bought it with the idea that it would provide insight into good (web) application design. However, after reading a couple of chapters, it is apparent that web sites, those that provide content, and web applications, those that do something, are different animals.

The book is fine, and is a recommended read for anyone new to web design. Unfortunately for me, Mr. Krug hasn't provided any new ideas that I haven't seen, tried, or evangelized at some point. It's easy to read with good examples; follow the guidelines in the book and your web site won't suck.

Time to Kill the Relational Model

It’s time to kill the relational database model. Well not the relational model per se, but rather the practice of building systems by developing relational schemas first.

This is quite the revelation for me. When first I discovered relational database design over three decades ago I thought it was religion. The simplicity of tables and relationships represented freedom from the rigid hierarchical structures of the day. Since then, relational design and its’ normal form remained a constant foundation for the systems I constructed.

But during a recent project I tired of the burden that relational design placed on our development effort. In particular, a tight adherence to rules of normalization and relational philosophy stifled our opportunity for nimble software engineering. What on paper looked like a method of enforcing integrity became complex morass of surrogate keys and multiple joins.

Fundamentally my complaint is over the complexity that normalization brings; an odd turn considering that relational databases were popularized on simplicity of tables. Frankly, though, there is nothing intuitive about joining tables; and the more joins the more complexity.

While relational databases are not (and should not) go away, there are flaws with some relational theory that makes writing software difficult. For example…

• Normalization produces a lot of tables. A lot of tables translate into a lot of joins. A lot of joins is a red-flag for complexity. When queries frequently have three or more joins, the schema is probably overly complex.
• Relational theorists discourage use of null foreign keys. The only way to accomplish this goal is to introduce a link table for the one to many relationships in the model. This introduces two problems, first, the extra table adds an additional (and unnecessary join). And second, it is harder to determine if a row in one table is related to a row in the other.
• Using data to determine state (or status) introduces complexity. While not necessarily a tenant of relational models, some designers prefer to compute the state of an item on-the-fly based on data in the tables. This is easy enough when all the data needed to determine the state are stored on the same table row. Unfortunately it is frequently necessary to scan additional tables on multiple rows to calculate the status.

A better method of system design starts with a design of objects. The database schema, then, is modeled to keep these objects intact. In this way, the database becomes nothing more than the repository for data at rest. The validity of the data is managed by methods within the objects and not by referential integrity or database triggers. Think of this as object first, data last design; there OFDL, I just coined a new acronym.

Why Digital Talent Doesn’t Want To Work At Your Company | Fast Company

I like this blog post so much that I am re-posting it in it's entirety.


Why Digital Talent Doesn’t Want To Work At Your Company | Fast Company:

Why doesn't digital talent want to work at your company? It’s not because you’re a consumer packaged goods company, rather than Google. It’s not because you’re in Ohio instead of Silicon Valley. It’s not because your salaries are too low, or because you don’t offer free food and laundry services.

It’s because you’re not providing them the right opportunity. The talent you want would be happy to work in an un-air-conditioned garage in New Mexico if it meant the chance to change the world.

This, the opportunity to do great things, to make a real difference, is what drives most digital talent--whether they’re developers, designers, producers, marketers or business folks. 

Most companies don’t offer this, so they skip your company and work somewhere that’s more innovative and exciting. End of story. But the good news is that you can offer them something exciting and great. The promise of changing a giant, behind-the-times organization into an Internet-savvy business is an incredibly exciting challenge and a big way for ambitious people to make an impact.  

But it takes more than lip service to make the sale. Job candidates and new hires with digital chops must truly believe in the company’s dedication to digital transformation and they must see that they are empowered to make this change. Trouble is, many big businesses aren’t structured to deliver on this type of opportunity. The attributes of a soul-crushing, Sisyphean, anti-digital workplace run deep.  

Digital talent won’t want to work at your company if:  

• Every element of their work will be pored over by multiple layers of bureaucracy. Even if that’s how the rest of the company operates, it can’t spill into the digital department. In a technology environment, new products and businesses spring up daily and a new endeavor can go from conception to launch in a matter of months. Reining in the momentum will be read as inaction and a clear signal the company isn’t willing to grasp the new way of the world.  
• Mediocre is good enough. While clocking out at 5 p.m. is attractive to some, it will discourage digital talent. They want to be expected to do something great. They want to be pushed. They care about their work. Their leadership, and those they rely on to get things done, must match their appetite for success.   
• Trial and error is condemned. The freedom to try out new ideas allows employees to take initiative, make decisions, and learn from their mistakes. It also demonstrates an attractive and inspiring entrepreneurial spirit.
• Your company is structured so it takes a lifetime to get to the top, and as such there are no digital experts in company-wide leadership positions.Digital talent--often in their 20s and 30s--need to see a clear path for uninhibited career development that’s based on merit, not years spent, and that’s beyond the confines of the digital department. If they don’t, they won’t see a reason to stay with the company in the long term.  
• Your offices are cold, impersonal and downright stodgy. It may sound like it conflicts with the “you don’t need to be in Silicon Valley point,” but appreciate the nuance. A traditional office layout is designed to communicate power among certain individuals and barriers between departments. This does not support the collaborative ethos which is intrinsic to the web. Companies should do everything possible to provide the digital team friendlier, open office space. A location in a hip, young neighborhood (which surely exists in every mid- to large-sized city) is also a big plus.  

When all of these digital-talent deterring points are addressed, company leadership has effectively and proactively demonstrated the company’s dedication to a digital transformation. It is at this time that their words, a broadly communicated firm stance on the significance of the company’s digital goals, will make the most impact. Without this conspicuous top-down support, politics in the organization or simply one influential disbeliever can hinder the effort, limit the extent of digital integration possible, and discourage valuable employees.  

You need them more than they need you. Demand for their services is so high, they can afford to be finicky. If they don’t like where they’re working, another firm with a more attractive culture and more grand opportunity will quickly swipe them up. That could be your company. But it could just as easily be someone else.

Adapted from Users Not Customers: Who Really Determines the Success of Your Business(Portfolio), by Aaron Shapiro, CEO of HUGE, a digital agency that helps companies including PespiCo, Comcast, Target, HBO, and Unilever reimagine how they interact with their customers and manage their business in the online economy. Visit aaronshapiro.com.

'via Blog this'

Thoughts on Safari (browser)

I just downloaded and installed the Safari 4 browser from Apple and it is, well, ugly! The ugliness is particularly bad on the rendering of its' tabs. The tabs are integrated into the window title bar with clumsy 3-D affects. It also eschews the former brushed aluminum theme for drab beige. I find this particularly disturbing since the software comes from one of the few technology companies that actually pays attention to design.

I was able to mitigate the ugliness a little by hiding the menu, bookmarks, and status bars. However, there's no changing the tabs. At first I rationalized the look in two ways. One, the software is beta so the appearance of the UI may change And two, Apple may be taking digs at Microsoft's Windows XP operating system. On further thought, neither makes sense as Apple should never risk its' reputation for design by releasing ugly software, especially on the most common desktop OS in the world.

That's not to say that Safari isn't a great web browser, it is innovative on a few fronts. First among these is speed, but does the average web surfer notice the speed of their browser? Maybe not, but the speed of Safari allows its' top page and history features to display and scroll pages. This works much like the CoolIris add-in (for IE and Firefox). I've only had Safari for a couple of hours, and I've already found the history search very useful.

The browser wars are heating up again and it's getting interesting. Internet Explorer 8, Chrome, FireFox, and Safari are all great software each with a compelling story. As for me, I use Explorer for all our intranet applications and Chrome as my default browser.

Atlantis found?

CrunchGear posted a blog indicating that Atlantis has been found. Sure, the source is a British tabloid, but apparently the ancient city was found using Google Earth.

While Atlantis is interesting in and of itself, the real story here is the technology. Google has been collecting data on the ocean floors using sonar. Impressive as it is that Google hired drivers to snap pictures of our major cities, it is even more incredible that they have hired boats to gather data about the ocean floor and stitch that data into their maps.

Don’t think of BI as an all-in-one solution

I recently had a short conversation on Business Intelligence with one of my peers. I tried to explain the premise that a Business Intelligence application in our industry (Health Care) should not be a one-size-fits all solution. Instead the technology should be tailored to the types of questions that it will need to answer most frequently.

When he claimed "of course it has to be able to answer any question, otherwise we could just write queries," I realized I failed to make my point. He is not the first person I've met to have this opinion. In fact, the opinion is pretty pervasive among my peers; and it is wrong.

Our Business Intelligence solution is a textbook case highlighting this point. Its' saga is a story for another day, but in our attempt to make it very flexible we failed to make it strong. That is, there's no limit to the reports you can create, but it's not great at answering any particular question.

I liken this to the difference between a hammer and a Swiss Army Knife. A hammer is great at driving nails, better than any other tool for this task. It also happens to be pretty good at removing nails too. A Swiss Army knife can do a lot of things from clipping nails to opening cans. But it's not particularly good at any of them.

The real beauty of a hammer, though, is the other things that it can do pretty well. In fact, if put to the test, it isn't hard to come up with at least as many tasks as can be done with a hammer as with the Swiss Army Knife. It can be a door-stop, a paper-weight, a meat tenderizer, a garden shovel, and more. Sure, it's lousy at tightening screws, but it can really drive nails.

Don't get me wrong. There's a place for firms that build all-in-one software. In fact, my former employer Information Builders is one such firm. My current BI solution is built on a MicroStrategy platform, another Swiss Army Knife vendor. Vertical solutions, like our health care application, need to be targeted; they need to be really good at particular questions.

Unfortunately, many designers of Business Intelligence solutions try to make Swiss Army Knives when they really need hammers. And given a good hammer and an innovative user, there will soon be many other tasks suitable for the tool.

Somebody agrees with my Chrome prediction

When Google introduced its' Chrome browser, I posted a short article on my thoughts (see Thoughts on Google Chrome). One of my conclusions was that the browser which would most feel competitive pressure would by FireFox.

"...If my browsing patterns are common, though, then Firefox will suffer more. In the end, Firefox and Chrome may just duke it out..."

Today I read an article with a similar conclusion. I'm happy that someone out there agrees.

The article, though, overlooks the primary reason why IE will remain the dominate browser; the fact remains that a huge library of browser based applications are not cross-browser (case in point, my team's products). This software is not aimed at the consumer or deployed on the World Wide Web. Instead the applications are deployed on corporate intranets around the world, assuring that IE remains the browser of choice for IT departments.

More web nonsense I don't need but will probably sign-up for anyway

Google has recently announced Latitude. This service allows you to track your friend's location using Google Maps. At first blush, I would categorize this service much like Twitter, as one of the dumbest services on the Web.

I'm a bit of a hypocrite when it comes to Twitter. I use it to update my Facebook status from my cell phone. As yet, though, I don't see how Latitude will provide a similar benefit and I don't think my friends will care about my exact location. Maybe my boss can use it to see how freely I come and go.

Latitude smacks a little like Big Brother. Google is famous for its' don't be evil mantra, but it this type of information will provide a lot of temptation for misuse. Privacy advocates will hate the service. In the end, joining and sharing the information will be voluntary, so I guess users will make their own trouble.

As for me, I'll kick the tires a bit, but you can bet that the service will be turned on my kids phones.

Some cool green transportation

An RSS from Fast Company just popped into my inbox. It describes the coolest electric vehicle I've seen. This is no Prius dork-mobile; no it's a hot super-bike capable of using it's electric motor for mind numbing acceleration. The bike is from Mission Motors, but go to Fast Company for the low-down.

Engaging the entrepreneurial power of employees with a venture capital model

The entrepreneur in us sees opportunities everywhere we look, but many people see only problems everywhere they look. The entrepreneur in us is more concerned with discriminating between opportunities than he or she is with failing to see the opportunities - Michael Gerber, author, entrepreneur

This year I participated, in an innovative project as we prepared for review by an internal leadership team. As of this writing, the project is not approved, but the financials are very strong and the technology is very low risk. By the time the project is reviewed by the board later this year, my firm will have spent nearly 18 months, hundreds of man-hours, and tens of thousands of dollars. Yet the work product produced is merely a document and an idea. There is no code and no proof of concept.

The procedure is at best a difficult process; at its’ worst an impossible process. There should be little doubt that the net effect of the process is to inhibit growth through innovation. It is, in fact, a barrier to entry for teams with go-to-market ideas.

Industry is full of companies with high profile examples of innovation. Firms like Google, Pixar, GE, and Apple show that a culture of innovation can bring success. This paper describes how any firm can evolve into a venture driven company that empowers employees with an entrepreneurial spirit.

The goal is plain and simple: achieve explosive growth through a culture of innovation.

To foster this culture, I suggest a model based on capital markets for startup companies. There are three aspects to this proposal:

• The corporation acts as a Hedge Fund by allocating funds for investment and measuring  aggregate results;
• Senior executives function as Venture Capitalists by soliciting ideas and managing a portfolio of investments; and
• Employees become the Entrepreneurs championing their ideas and finding resources to execute a project.

 

The critical ingredient is getting off your butt and doing something. It's as simple as that. A lot of people have ideas, but there are few who decide to do something about them now. Not tomorrow. Not next week. But today. The true entrepreneur is a doer, not a dreamer - Nolan Bushnell, founder of Atari and Chuck E. Cheese's

The Hedge Fund

A hedge fund is a private investment fund whose managers can make speculative investment and participate in profits from money invested. The company can function much like a hedge fund by structuring projects as a portfolio of investments. The managers of the “hedge fund” are the executives who normally vet within for the company.

The corporation, then, decides how much to invest in the fund. Each year these funds are set aside for innovation and it becomes the responsibility of the hedge fund managers to assure the funds are invested, and that the portfolio achieves an appropriate return.

As the Hedge Fund, the company would:

• Invest in the fund by allocating monies for innovation. An investment to the innovation fund each year;
• Appoint  executives to function as the Venture Capitalists;
• Set an ROI threshold for the fund

Taking Risk

I have not failed. I've just found 10,000 ways that won't work - Thomas Edison, inventor and scientist

A great percentage of start-ups fail. Hedge funds that specialize in innovation spread their investments over a wide range of projects with the understanding that a small number of winners will earn a return for the entire portfolio. Risky projects will be an integral part of the firm’s portfolio.

Like true VC hedge funds, the firm can mitigate risk by funding projects that already have some track record of success. This track record could be a successful competitor, a custom built technology, or a joint venture with a client.

Exit Strategies

Unlike the holdings in venture capital hedge funds, the company’s investments generally do not end with a liquidity event (sale of the company or IPO).Instead successful investments will result in product lines or business that generate revenue and profit. Less successful projects are absorbed into existing businesses or closed outright.

It should be noted that there is the possibility of a liquidity event. Although these may be rare, some projects could be spun off as independent companies. In such cases the firm would act as a private equity fund, holding a stake in the company until an appropriate liquidity event occurs.

Venture Capitalists

Since one fails often, address markets that make it worthwhile when one does succeed. Vinod Khosla, co-founder of Sun Microsystems

Executives that normally vet for capital investment will serve as the venture capitalists in this model. As VC, each executive is expected to oversee a portfolio of projects and to assure that their funds are invested. Each VC would have a specialty where they would focus their investments. It would be expected, though, that they would invest outside their area on occasion.

The VC has four primary roles:

• Invest a share of the fund in projects of their choosing;
• Provide oversight of the projects in their portfolio;
• Mentor the entrepreneurs assigned to the projects in their portfolio; and
• Arrange exit strategies for their projects.

Key to the success of the hedge fund is the oversight and mentoring role of the VC. Unlike an “approve-and-forget” capital investment model, VCs of the hedge fund share responsibility for the success of their portfolio. This oversight keeps entrepreneurs accountable to accomplish milestones and meet their financial goals.

VCs solicit, review, and select projects based on the business plan from the entrepreneur. The VCs is empowered and required to invest his share of the fund.  VCs will be expected to accomplish the appropriate due diligence, but being empowered will make the decision process move quickly. With empowerment comes accountability, and the VC will be expected to make rational decisions based on standard venture criteria such as, experience of management, amount of existing business, size of the market, first-in status, and the competitive landscape.

Like true VCs, the managers in the firm’s hedge fund will collaborate on some projects. Some of the VCs will assume an “angle investor” role by making small investments in a large set of projects in their early stages. As angle projects progress, the VC may decide to invest additional rounds of financing. Also, several VCs may invest together in a project, thereby sharing the risk, or implementing a large scale innovation.

VCs shall participate is the financial success of their portfolio. A bonus would be paid based on the success of the projects in the portfolio.

I never perfected an invention that I did not think about in terms of the service it might give others... I find out what the world needs, then I proceed to invent -Thomas Edison, inventor and scientist

Intrapreneurs

Some people dream of great accomplishments, while others stay awake and do them – Anonymous

Entrepreneur : one who organizes, manages, and assumes the risks of a business or enterprise.

Intrapreneur : one who organizes, manages, and assumes the risks of an idea within a business or enterprise.

The entrepreneurs are the employees that champion an idea (intrapreneur is a term made up for this white paper). Entrepreneurs are passionate about an idea, and work to see it through to its completion. This includes building prototypes, writing business plans, seeking funding from VCs, and managing the project day-to-day.

Much of the motivation and reward of these employees is intrinsic and non-financial. This includes the excitement of working on a new and innovative idea. In addition, once funding is approved, the employee will have effectively transferred into a new position with the company. Like the VC, however, the entrepreneur will be rewarded based on the financial success of the project or its exit strategy.

The entrepreneur has the following roles:

• Develop a business plan;
• Seek and obtain funding from the company VCs;
• Build prototypes and proof-of-concepts for the project;
• Obtain resources needed to complete the project;
• Manage the project, day-to-day; and
• Report progress and financial results to the VC

 

It should be expected that projects are present by a team of employees. In these cases, one member of the team shall be designated as the entrepreneur. It is likely that the entrepreneurs are employees that already hold management or leadership positions (but that are not a requirement).

Mitigating Personal Risk

Entrepreneurs assume a great deal of personal risk with regard to the security of their career. Although positions cannot be held open for the entrepreneur pending completion of their project, the corporation should guarantee that a position will be made available regardless of the success or failure of the project. Truly motivated entrepreneurs, however, do not consider failure, and will pursue the idea under the assumption of success.

The Business Plan

Entrepreneurs must submit a business plan to obtain funding for their projects. Business plans should follow common standards for start-up plans. At a minimum, a business plan shall include:

• Description of the Idea;
• Marketing Plan;
• List of existing customers
• List of target customers
• Size of the market
• Pricing
• Competitive Analysis
• List of competitors
• Advantages and Disadvantages
• Operational Plan
• Management Team
• Personnel
• Technology
• Financial Plan
• Start-up Expenses
• 3 Year Pro-Forma

 

Advantages of the Hedge Fund Model

This approach to funding invention, and building a culture of innovation has distinct advantages over my company’s current model.

• The current procedure in of itself is an expensive and labor intense process. It requires multiple stages of presentation and approval. The current process is self-regulating, scaring away employees that are intimidated by the process regardless of the quality of their idea.
• An innovation provides a method of easily submitting an idea, but except for quick rejections, it routes ideas through the same burdensome approval process.
• The hedge fund model is simple; write a business plan and present it to a VC.
• Our current procedure is slow.  It will be very difficult to have first-in status if seeking funding through the current process. Real-time Fraud and Abuse, for example, has been in-process for months.
• The hedge fund model is fast.
• Where our current procedure is focused on preparation and planning during the approval process, hedge fund is focused on execution of the idea.
• Hedge fund rewards entrepreneurs and VCs for taking risk and accomplishing goals.
• Hedge fund provides a tangible portfolio of innovation and R&D.
• Hedge fund is based on tried and true practices of bring an idea to market.

 

Twenty years from now you will be more disappointed by the things that you didn't do than by the ones you did do. So throw off the bowlines. Sail away from the safe harbor. Catch the trade winds in your sails. Explore. Dream. Discover -Mark Twain, author

Twitter gives up on Instant Messaging (for now)

Twitter, one of Web 2.0's most pointless sites, is giving up on their Instant Messaging capability. According to their status page, Twitter is changing the classification of the feature from something needing a fix to something they will build. Apparently their implementation of the feature is neither scalable nor reliable.

Twitter's existence is based on their invention of "micro-blogging." Micro blogs are short notes posted in response to the question "what are you doing." Beyond the concept of a micro-blog Twitter makes it easy to submit updates through their web site or via an SMS message. Instant messaging is simply another way to post the tiny blogs.

The relevance of Twitter has to come into question. Certainly Facebook's news feed serves the same purpose, and is much more common. In fact, most of my Twitter posting are made through the Twitter Facebook application. Maybe it's me, but I believe the real value to Twitter is it's ability to post comments via several easy to use methods. Instant messaging is one of those sources and should be an urgent capability to add. Otherwise the company will lose any relevance to others providing a similar service as part of a greater offering.

Thoughts on Google Chrome

A couple of years ago, a friend of mine said "everything Google does is excellent". He was showing me Google Maps, which at that time, was revolutionary. Until then, I considered Google another search vendor of Yahoo and Alta Vista ilk. Shortly thereafter I signed up for Google Mail (remember when it was by invitation only), then Google Talk, and later Google Docs.

Although I like Google Docs a lot, using it made me realize that not everything from Google is excellent. And that sums up my opinion about Google's new browser, Chrome. While I like it, and am using it a lot, it's not excellent…yet.

I downloaded and installed Chrome shortly after it was made publicly available this week. Like everything that Google does, the download and installation was fast and automatic. Getting started is easy. Regardless of their claims, Chrome is really just another browser. But true to the Google brand, the browser excels at search. Sure the integrated address and search bar is nice, but IE and firefox let you search from the address bar too. The search feature I really like is the within a page; I like the way Chrome highlights hits on the scrollbar.

Two days is not a lot of time to work with a program, and being a new application in a mature market, Chrome is expected to have flaws. Here some of the issues that I found:

• Sites that perform browser sniffing don't render properly. Hotmail, for instance, only supports its' classic html mode. And our FogBugz issue tracking software refuses to run its' Wiki in Chrome.
  
• Facebook applications behave oddly, in particular I found that links and buttons, such as "Allow" or "Ignore" fail.
  
• Our web applications developed by my teams do not render well, but then they don't render well in FireFox either. We remain a Microsoft shop.
  
• Silverlight applications do not run in Chrome, so Google surfers can't see pictures of my family.
  
• There's no history list, at least not without typing the first few characters of a web site.
  
• There's no button to display a home page; well Ok, there's a setting but it's off by default.
  
• I can't (and maybe it's just me) get back to the frequent-page thumbnails without opening a new tab. I want to make this my home page.
  

I find it interesting that Google chose to enter the browser market. Do we need another browser? It's only been a couple of years since Internet Explorer was the only real player. If Chrome catches on there will be four; IE, Firefox, Safari, and Chrome (nah, I don't count Opera). Personally, I find myself gravitating to Chrome despite its' flaws and am considering making it my default browser. Although I use Internet Explorer a bit less I find that I never open Firefox anymore (I have Safari for Windows, but really, who uses that?); this after only two days with Chrome.

The competitive target of Chrome is Internet Explorer, and certainly the Chrome comic book takes shots at IE weaknesses. If my browsing patterns are common, though, then Firefox will suffer more. In the end, Firefox and Chrome may just duke it out for the (tiny but vocal) anti-Microsoft crowd. I heard a rumor that Google's entry into the browser space is a precursor to their entry into PDA's with Android. In much the same way that Apple leveraged Safari on the iPhone, Google will leverage Chrome. And on a tiny cell phone screen Chrome's quirkiness will be unimportant. The cell phone angle is intriguing, but Google's open platform approach will produce odd devices and applications that will seem crude when compared to Apple's cohesive offering. If nothing else, Google is making the software industry fun again.