Sunday, January 5, 2014

Nexus of Five - Social, Cloud, Mobile, Big Data and Artificial Intelligence (5F) Intro

Traditional Companies planning to go Social and to leverage the amount of data generated internally and externally - are they really Ready to be able to manage such data ?

Managing entire data layer for a Large Insurer , thought of how the Infrastructure needs to be designed for better management When My_Customer embarks on social strategy.

Gear up their Infrastructure for such needs Link
This means terabytes’ of storage, if not more. Additional storage means more cost to already huge company’s spend in Storage ,  which we have been showing some innovative ideas like compression etc to save and also have Stringent process which almost takes three months to get a chunk of storage – more Storage admins – more storage failures Note:  they do have sizable Hardware failures every other day and more and more storage is going to just aggravate the issue if not taken care

Challenge : Data needs to be stored in huge quantities and also available (with performance) at a Low Cost (Mutually exclusive unless taken proper measures)

Answer :-  Raid Arrays - not exactly , they could add to problem of Cost being replicated 5 times. Larger chunk for efficient usage of power would mean lot of time to replace full parity - erasure codes
Reed-Solomon erasure codes were originally used as forward error correction (FEC) codes for sending data over an unreliable channel, like data transmissions from deep space probes. Using erasure codes, a piece of data can be broken up into multiple chunks, each of them useless on their own, and then dispersed to different disk drives or servers. At any time, the data can be fully reassembled with a fraction of the chunks, even if multiple chunks have been lost due to drive failures.
Clever safe - dispersal coding uses erasure codes with location info.
  • Width- The number of pieces you generate.  
  • Threshold - minimum number needed to put it back together.
  • Width – Threshold determines its reliability. The highest amount of reliability you can get with RAID is dual parity. You can lose two drives. That's it. With our solution, you can lose up to six
An Erasure code is scalable as it is software-based technology.  When a drive fails or goes offline, storage infrastructure is able to mark it as unavailable and route data around it while recovering data on that drive transparently- Instead of an "all hands on deck" situation.

Reliability has some thoughts, How about Performance ?              

                     ·     Database Structure Link
o   Partition….
           ·     Database Search Patters

Parallel to technology how the Business  strategy should be for the Nexus of Force ( Social, Mobile, Cloud, AI and BigData)
o   How they should approach Social presence
o   What BIG Data initiatives they should embark? BIG Data in 2014 is on trough wave of disillusionment
How they could do IBO
o   How they should leverage Cloud (Private and Public) not to recreate available resources outside ?
§  Just done a pricing for a Private cloud unit based pricing
o   How to leverage Machine learning help the same in general and Insurance in Specific
§  List of use cases ?

§  Important as data increases Artificial Intelligence is built in conjunction else resources needs will be needed heavily