TABLE OF CONTENT

ABSTRACT …………………………………………………………………………………………………………………………………….. 7
BACKGROUND ………………………………………………………………………………………………………………………………. 8
BIG DATA DEFINITION, HISTORY AND BUSINESS CONTEXT …………………………………………………………………… 9
WHY IS BIG DATA RESEARCH IMPORTANT? ……………………………………………………………………………………… 11
BIG DATA ISSUES …………………………………………………………………………………………………………………………. 12
BIG DATA OPPORTUNITIES ……………………………………………………………………………………………………………. 14
Use case- US Government………………………………………………………………………………………………………………………. 16
BIG DATA FROM A TECHNICAL PERSPECTIVE ……………………………………………………………………………………. 17
Data management issues ……………………………………………………………………………………………………………………….. 18
1.1 Data structures …………………………………………………………………………………………………………………. 19
1.2 Data warehouse and data mart ………………………………………………………………………………………….. 21
Big data management tools ……………………………………………………………………………………………………………………. 23
Big data analytics tools and Hadoop ………………………………………………………………………………………………………… 24
Technical limitations relating to Hadoop ………………………………………………………………………………………………….. 26
1.3 Table 1. View of the difference between OLTP and OLAP ………………………………………………………… 29
1.4 Table 2. View of a modern data warehouse using big data and in-memory technology ……………… 30
1.5 Table 3. Data life cycle- An example of a basic data model …………………………………………………….. 31
DIFFERENCES BETWEEN BIG DATA ANALYTICS AND TRADITIONAL DBMS ……………………………………………… 32
1.6 Table 4: View of cost difference between data warehousing costs in comparison to Hadoop ……… 33
1.7 Table 5. Major differences between traditional database characteristics and big data
characteristics …………………………………………………………………………………………………………………………… 34
BIG DATA COSTS- FINDINGS FROM PRIMARY AND SECONDARY DATA …………………………………………………. 35
1.8 Table 6: Estimated project cost for 40TB data warehouse system –big data investment …………….. 38
RESEARCH OBJECTIVE …………………………………………………………………………………………………………………… 41
RESEARCH METHODOLOGY …………………………………………………………………………………………………………… 42
Data collection ……………………………………………………………………………………………………………………………………… 44
Literary review ……………………………………………………………………………………………………………………………………… 46
Research survey ……………………………………………………………………………………………………………………………………. 47
1.9 Table 7: Survey questions …………………………………………………………………………………………………… 48
SUMMARY OF KEY RESEARCH FINDINGS ………………………………………………………………………………………….. 53
RECOMMENDATIONS …………………………………………………………………………………………………………………… 57
Business strategy recommendations ……………………………………………………………………………………………………….. 57

6
Technical recommendations …………………………………………………………………………………………………………………… 58
SELF-REFLECTION …………………………………………………………………………………………………………………………. 59
Thoughts on the projects ……………………………………………………………………………………………………………………….. 59
Formulation …………………………………………………………………………………………………………………………………………. 63
Main learnings ……………………………………………………………………………………………………………………………………… 64
BIBLIOGRAPHY …………………………………………………………………………………………………………………………….. 66
Web resources ……………………………………………………………………………………………………………………………………… 67
Other recommended readings ………………………………………………………………………………………………………………… 68
APPENDICES………………………………………………………………………………………………………………………………… 69
Appendix A: Examples of big data analysis methods ………………………………………………………………………………….. 69
Appendix B: Survey results ……………………………………………………………………………………………………………………… 72