Category: Unsupervised Machine Learning

  • Time Series Data Imputation, Interpolation & Anomaly Detection

    Time Series Data Imputation, Interpolation & Anomaly Detection

    The post compares popular time series data imputation, interpolation, and anomaly detection methods. It explores the challenges of missing data and the impact on processing, analyzing, and model accuracy. The study performs data-centric experiments to benchmark optimal methods and highlights the importance of imputation for time series forecasting. It provides practical strategies and techniques for…

  • Malware Detection & Interpretation – PCA, T-SNE & ML

    Malware Detection & Interpretation – PCA, T-SNE & ML

    This post discusses the application of PCA, T-SNE, and supervised ML algorithms for malware detection using a benchmark dataset. Techniques such as Logistic Regression, SVC, KNN, and XGBoost are implemented, achieving high performance metrics. Results show potential for improving malware detection using ML while reducing false positives and enhancing cyber defense.

  • Sales Forecasting: tslearn, Random Walk, Holt-Winters, SARIMAX, GARCH, Prophet, and LSTM

    Sales Forecasting: tslearn, Random Walk, Holt-Winters, SARIMAX, GARCH, Prophet, and LSTM

    The data science project involves evaluating various sales forecasting algorithms in Python using a Kaggle time-series dataset. The forecasting algorithms include tslearn, Random Walk, Holt-Winters, SARIMA, GARCH, Prophet, LSTM and Di Pietro’s Model. The goal is to predict next month’s sales for a list of shops and products, which slightly changes every month. The best…

  • Weather Forecasting & Flood De-Risking using Machine Learning, Markov Chain & Geospatial Plotly EDA

    Weather Forecasting & Flood De-Risking using Machine Learning, Markov Chain & Geospatial Plotly EDA

    Foto door Pok Rie The most frequent natural disaster in the world, flooding affects hundreds of millions of people and kills between 6,000 and 18,000 people annually. Moreover, climatic change has many consequences as surge in frequency of rainfalls potentially enhance the rate of flooding. In this environmental science project, we will explore real-time weather…

  • Dividend-NG-BTC Diversify Big Tech

    Dividend-NG-BTC Diversify Big Tech

    SEO Title: Can Dividends, Natural Gas and Crypto Diversify Big Techs? The objective of this project is to implement several basic Quant Trading (QT) optimization ideas by analyzing tech growth, dividend stocks and highly volatile assets (commodities and crypto) in terms of risk/return optimization scenarios and (potential) diversification options. We will use the financial APIs…

  • Anomaly Detection using the Isolation Forest Algorithm

    Anomaly Detection using the Isolation Forest Algorithm

    The post describes the application of Isolation Forest, an unsupervised anomaly detection algorithm, to identify abnormal patterns in financial and taxi ride data. The challenge is to accurately distinguish normal and abnormal data points for fraud detection, fault diagnosis, and outlier identification. Using real-world datasets of financial transactions and NYC taxi rides, the algorithm successfully…

  • Returns-Volatility Domain K-Means Clustering and LSTM Anomaly Detection of S&P 500 Stocks

    Returns-Volatility Domain K-Means Clustering and LSTM Anomaly Detection of S&P 500 Stocks

    This study aims to implement and evaluate the K-means algorithm for ranking/clustering S&P 500 stocks based on average annualized return and volatility. The second goal is to detect anomalies in the best performing S&P 500 stocks using the Isolation Forest algorithm. Additionally, anomalies in the S&P 500 historical stock price time series data will be…

  • Real-Time Anomaly Detection of NAB Ambient Temperature Readings using the TensorFlow/Keras Autoencoder

    Real-Time Anomaly Detection of NAB Ambient Temperature Readings using the TensorFlow/Keras Autoencoder

    The content covers a detailed guide on implementing anomaly detection in time series data using autoencoders. The tutorial utilizes Python and real-world temperature dataset from Numenta Anomaly Benchmark (NAB). Following the Python workflow, the algorithm imports required libraries, performs anomaly detection, and visualizes anomalies. A trained autoencoder model identifies anomalies, with Precision, Recall, and F1…

  • Improved Multiple-Model ML/DL Credit Card Fraud Detection: F1=88% & ROC=91%

    Improved Multiple-Model ML/DL Credit Card Fraud Detection: F1=88% & ROC=91%

    In 2023, the global card industry is projected to suffer $36.13 billion in fraud losses. This has necessitated a priority focus on enhancing credit card fraud detection by banks and financial organizations. AI-based techniques are making fraud detection easier and more accurate, with models able to recognize unusual transactions and fraud. The post discusses a…

  • Unsupervised ML, K-Means Clustering & Customer Segmentation

    Unsupervised ML, K-Means Clustering & Customer Segmentation

    The goal of this post is to describe the most popular, easy-to-use, and effective data-driven techniques that help organizations streamline their operations, build effective relationships with with individual people (customers, service users, colleagues, or suppliers), increase sales, improve customer service, and increase profitability. Customer Segmentation (CS) is a core concept in this technology. This is the…

  • Effective 2D Image Compression with K-means Clustering

    Effective 2D Image Compression with K-means Clustering

    The post explores the application of the K-means clustering algorithm, a popular unsupervised Machine Learning algorithm, for image compression. By segmenting 2D images into different clusters, the algorithm effectively reduces storage space without compromising on image quality or resolution. It also demonstrates the application of this approach through a case study, where optimal results were…