feature engineering

In machine learning, feature engineering is an important step that determines the level of importance of any features from the data. In one of our articles, we have seen that ridge regression is used to get rid of...

We are excited to announce the integration of Tecton’s enterprise feature store and Feast, the popular open source feature store, with Snowflake. The integration, available in preview to all Snowflake customers on...

To maintain the reliability of the machine learning models, we need to improve their explainability and interpretability. There should be a proper reason that we can give while the model is making the predictions....

An alternative approach to standardizationPhoto by Filip Mroz on UnsplashOne of the most common feature engineering techniques for numerical features is to standardize those features. This is often accomplished either...

Three ways to bin numeric featuresPhoto by frank mckenna on UnsplashBinning numerical features into groups based on intervals the original value falls into can improve model performance. This can occur for several...

One-hot encoding in Python and on the data warehousePhoto by Burst on UnsplashWhile most machine learning algorithms only work with numeric values, many important real-world features are not numeric but rather...

Part 1: Data PreprocessingImage by Pete Linforth from Pixabay“Applied machine learning is basically feature engineering”— Andrew NgThe era of Deep Learning has popularized the approach of end-to-end machine...

Functions and DAGs: Hamilton, a General Purpose Micro-Framework for Pandas Dataframe GenerationAn approach for preventing your code from getting too complex to navigateCreating dataframes is easy, but managing the...

Feature engineering plays a crucial role in many of the data modelling tasks. This is simply a process that defines important features of the data using which a model can enhance its performance. In time series...

Joining data on the modern data stackPhoto by Tim Foster on UnsplashWhen working in data science, rarely will a data scientist be able to find all of the necessary data for their project in a single file. Even if the...

Imputation at scalePhoto by Sigmund on UnsplashSince the majority of machine learning algorithms do not handle null values, missing value imputation is a standard part of feature engineering pipelines. There are...

Why “Big Data” Transformation and Feature Engineering is vital to ML successThis article will covers the following:1- What is a Data Product2- Challenges in Data Transformation3- Planning your Data Transformation4-...

Counter-intuitive geometric properties of high-dimensional spacesMany of our intuitions about distance, area, and volume break down in high-dimensional spaces. In AI/ML and data science, we work with high-dimensional...

Photo by Richard Horvath on UnsplashExploring the wave related features of an audio file for further use in Analytics and MLIn the rise of the Big Data era, we can collect more data than ever. Data collection is no...

SVM, PCA and HOG join forces to solve a Computer Vision problemPhoto by Hello I’m Nik on UnsplashIntroductionData and Machine LearningWhen it comes to talking about Machine Learning, it’s clear that it is the...

A brief exhibition of the power of simple feature engineering using house sale price data and a LightGBM model.Photo by Danist Soh on UnsplashA brief backgroundFeature engineering can mean different things to...

Geographic dataFeature engineering distance from geographic coordinatesWhen preparing data for a model, there may be a time where it’s useful to find distances between two locations. This post shows how to find the...