Big Data is a term used for the voluminous data collection and archiving that is happening with the advent of Internet, digital devices and volunteered data. Data sets are becoming so large or complex that traditional methods of storage, processing, applications are being rendered inadequate to deal with them. Challenge today is to cope with the wide variety of data capture, data formats and standards, data storage and archival, data synthesis and integrated analysis, data search, data sharing and transfer, data or information visualization, querying and taking the right decisions with the data available.
Data in Big data is universal – it includes spatial data also alngwith large volumes of other data on resources, events, individuals, social behavious, financial, insurance, aviation and many many other elements. The challenge is in bringing a good sync between the spatial character of data with the other non-spatial character of data. Big Data is triggering Data Analytics – predictive analytics, user behavior analytics, social analytics, econometrics or certain other advanced data analytics methods that extract value from data, and seldom to a particular size of data set. Analysis of data sets can find new correlations to predict economic trends, social behaviour patterns, spot business trends, identify disease and health streaks, identify locations, combat crime and so on.
Big Data grows rapidly – in part because they are increasingly gathered by cost-effective and numerous information-sensing devices such as mobile devices, aerial (remote sensing), UAS, satellites, software logs, cameras, microphones, radio-frequency identification (RFID) devices and wireless sensor networks. One issue for governments, enterprises and institutions is determining how and who should archive big-data. There are many areas where Big Data would be of value for many groups of people, governments and organisations.
C-SAG works on concepts of Open Data Initiative (ODI) that mandates and brings all data funded by public funds to be brought into a common and standardized Big Data platform is the order of the day. Such initiatives catalyze the data analytics eco system in society and for governance by enabling a variety of stakeholders to play a role in data-driven decision making.
C-SAG is developing standards for Big Data – especially on spatial big data for a national or state eneterprise of GIS and addressing technical aspects of spatial database standards, spatial frameworks, data formats and inter-operability, services for Big Data, archiving and storage methods for GIS data, efficient clustered search on spatial logic in Big Data and also overall framework for Data Analytics principles.