hadoop

Do you need a Cloud Management Platform for your company? OpenStack is a good choice of many organizations. This open-source platform enables companies to benefit from cloud computing and manage their resources. But is...

Apply Now Salary: 13 Mil - 23 Mil Address: P804A, 22 Láng Hạ, Quận Đống Đa, Hà Nội See map Posted: 27-03-2021 Expired: 26-04-2021 Job Description - Tham gia phân tích, thiết kế,...

With companies forced to adapt to a remote, distributed workforce this past year, cloud adoption has accelerated at an unprecedented pace by +14%  resulting in  2% or $13B above pre-pandemic forecasts for 2020 –...

Do you have a lot of text data that requires them to count the occurrence of every single unique word? If yes, you’ve got Hadoop’s back to process this ‘Big Data’ of yours.In this article, we’ll try our hands...

The first part of this two part blog provided an introduction to concepts in Kubernetes and Public Cloud that are relevant to stateful application management. We also covered how Kubernetes and Hadoop features were...

Authors: Dhiraj Hegde, Ashutosh Parekh, and Prashant MurthyAt Salesforce, we run a large number of HBase and HDFS clusters in our own data centers. More recently, we have started deploying our clusters on Public Cloud...

Growing demand for data governance, privacy and security tools needed to comply with an expanding list of national data-privacy rules continues to attract investors as more sensitive data shifts to the cloud. The latest...

Hey all!In this story let’s talk about LVM and how to integrate it with Hadoop for storage management.Let us practically understand it.For this, I am attaching the virtual storage of 20 GB to the slave node.Now start...

The Basics of HadoopHadoop is an Apache open-source framework written in java that allows distributed processing of large datasets across clusters of computers using simple programming models.First of all, what does it...

Photo by Glenn Carstens-Peters on UnsplashIntroductionThis article is based on my previous article “Big Data Pipeline Recipe” where I tried to give a quick overview of all aspects of the Big Data world.This...

Between Hadoop and the Cloud: Where Do We Go from Here?In a post-COVID-19 future, businesses must be prepared to respond and adapt rapidly. Is Hadoop still relevant?Image by AuthorIn the emerging world of 2021, big...

In a prior post we provided a terraform module for deploying Dataproc clusters with Kerberos and trust with MIT KDC for managing user/service account principals. While MIT KDC provides the ability to quickly get...

According to Gartner, over 90% of deployed data lakes will become useless as they are overwhelmed with information assets captured for uncertain use cases. This is indeed, an alarming situation. While companies are...

MapReduce is the heart of Apache Hadoop. The MapReduce algorithm has two parts: Map and Reduce. Mapping involves processing a large data set parallelly to generate <key,value> pairs. These <key,value> pairs are...

Deciding about Pyspark configuration parameters with the usage of YARN as a cluster management frameworkApache Spark is one of the most popular open-source distributed computing platforms for in-memory batch and stream...

Engineers at Netflix and Apple created Apache Iceberg several years ago to address the performance and usability challenges of using Apache Hive tables in large and demanding data lake environments. Now the data table...