As of late, everyone seems to be talking about cloud-based architecture and the need for various platforms including cloud management. What no one seems to be talking about is the huge growth of digital data stored across the world.
Big data and digital data have impacted cloud services around the world. Cloud is, and will continue to be, an important aspect of IT landscapes and is considered the cure-all for every IT issue out there. There are two major elements of all IT, the data and the logic working with the data. Everybody working with big data knows that in order to use huge levels of data, you must bring the processing of data to data, not the other way around. Processing from a distance can create bottlenecking which will cause performance to decrease practically to nothing as well as the functioning of that logic.
If you move your application server to the cloud but keep your database service at your place of business, when you perform an application that is latency-sensitive, the application server and database server will not work. This is already the situation with smaller levels of data and why many businesses are trying to adapt software to become less latency-sensitive by moving to the cloud. That said, you need to bring processing and data close to each other for large amounts of data to work. To deal with enormous amounts of data, you really need Hadoop and other architectures to deal with the problem of processing massive amounts of data.
It is expected that in a few years the world will store approximately 50ZB. While the Internet's capacity for moving data is on the rise, it's doing so at a much slower pace. Adding to that, the total internet bandwidth will reach around 2.5ZB on a yearly basis.
Should this information turn out to be true, there will not be enough internet bandwidth available to move even a fraction of the data anywhere. Also, 80% of all current bandwidth is currently used for streaming videos. Even if you have addressed your latency issues for massive data, there will still be an issue with your bandwidth.
If you are processing data in the same location that holds the data, there shouldn't be an issue. As the number of data increases, everyone is looking for cloud strategies and going to extremes by putting everything in the cloud.
While more and more people are getting into cloud-based platforms for the distribution of data, the amount of data you use is getting larger and larger. You will have to consolidate data and the processing in a single physical location.
Another solution is in the rise of colocation providers. They are providing large data centers with optimized internal traffic abilities where cloud providers and large cloud users are working together. What this means, you may be in the cloud but physically you are in the same space as your cloud provider. You want to run your logic on a data center where you will also have your own private data lake. All the data is local to the processing and data aggregation. It is possible that cloud providers will extend into your data centers but then again, colocation seems to be another possible solution for bandwidths and other issues regarding the growth of data.
If all seems doom and gloom, it's not. The lack of stability of data is actually very low. That said, just thinking you can distribute your work to a variety of different cloud providers can still be risky. If the data you are sending increases in volume, which it probably will if everyone combines their own data streams from Facebook, Twitter, etc and creates new streams.
It's important you create a strategy regarding the location of your data and processing, what you can and cannot isolate from other data.
Big data and digital data have impacted cloud services around the world. Cloud is, and will continue to be, an important aspect of IT landscapes and is considered the cure-all for every IT issue out there. There are two major elements of all IT, the data and the logic working with the data. Everybody working with big data knows that in order to use huge levels of data, you must bring the processing of data to data, not the other way around. Processing from a distance can create bottlenecking which will cause performance to decrease practically to nothing as well as the functioning of that logic.
If you move your application server to the cloud but keep your database service at your place of business, when you perform an application that is latency-sensitive, the application server and database server will not work. This is already the situation with smaller levels of data and why many businesses are trying to adapt software to become less latency-sensitive by moving to the cloud. That said, you need to bring processing and data close to each other for large amounts of data to work. To deal with enormous amounts of data, you really need Hadoop and other architectures to deal with the problem of processing massive amounts of data.
It is expected that in a few years the world will store approximately 50ZB. While the Internet's capacity for moving data is on the rise, it's doing so at a much slower pace. Adding to that, the total internet bandwidth will reach around 2.5ZB on a yearly basis.
Should this information turn out to be true, there will not be enough internet bandwidth available to move even a fraction of the data anywhere. Also, 80% of all current bandwidth is currently used for streaming videos. Even if you have addressed your latency issues for massive data, there will still be an issue with your bandwidth.
If you are processing data in the same location that holds the data, there shouldn't be an issue. As the number of data increases, everyone is looking for cloud strategies and going to extremes by putting everything in the cloud.
While more and more people are getting into cloud-based platforms for the distribution of data, the amount of data you use is getting larger and larger. You will have to consolidate data and the processing in a single physical location.
So What Does The Future Hold?
Many people are running around to minimize the need to move data. In the IoT world, there are many discussions regarding handling data locally where IoT devices and sensors are. Again, all that means is processing must be local as well. You can assume that you will not have the same level of computing power in a set of sensors than what you could do in big analytic setups. You can minimize data traffic but at the expense of how much you can compute.Another solution is in the rise of colocation providers. They are providing large data centers with optimized internal traffic abilities where cloud providers and large cloud users are working together. What this means, you may be in the cloud but physically you are in the same space as your cloud provider. You want to run your logic on a data center where you will also have your own private data lake. All the data is local to the processing and data aggregation. It is possible that cloud providers will extend into your data centers but then again, colocation seems to be another possible solution for bandwidths and other issues regarding the growth of data.
If all seems doom and gloom, it's not. The lack of stability of data is actually very low. That said, just thinking you can distribute your work to a variety of different cloud providers can still be risky. If the data you are sending increases in volume, which it probably will if everyone combines their own data streams from Facebook, Twitter, etc and creates new streams.
It's important you create a strategy regarding the location of your data and processing, what you can and cannot isolate from other data.
Big Data Vs The Cloud
Reviewed by thanhcongabc
on
December 21, 2017
Rating:
Reviewed by thanhcongabc
on
December 21, 2017
Rating:
No comments: