PyQuant News

Resources for developers using Python for scientific computing and quantitative analysis

You are here: Home / Archives for Data Science
Latest Resources in Data Science

Data Science

under Data Science

Reddit has been at the epicenter of one of the biggest movements in the world of finance, and although it seemed like an unlikely source of such a movement — it’s hardly surprising in hindsight.

The trading-focused subreddits of Reddit are the backdrop for a huge amount of discussion about what is happening in the markets — so it is only logical to tap into this huge data source.

When building a data extraction tool like this, one of the first things we need to do is identify what the data we’re extracting is actually about — and for that we will be using named entity recognition (NER).

under Data Science

In this article the author uses Reddit sentiment data to inform trading strategies. He derives market sentiment in two ways using the wallstreetbets subreddit:

  1. Collecting comments from daily discussion submissions then running the VADER sentiment model to assess overall daily positive/negative sentiment.
  2. Collecting all submission titles per day then assessing daily bullish/bearish sentiment using keyword analysis.

Introducing PyTorch Forecasting towardsdatascience.com

under Data Science

Forecasting time series is important in many contexts and highly relevant to machine learning practitioners. Take, for example, demand forecasting from which many use cases derive. Almost every manufacturer would benefit from better understanding demand for their products in order to optimise produced quantities. Underproduce and you will lose revenues, overproduce and you will be forced to sell excess produce at a discount. Very related is pricing, which is essentially a demand forecast with a specific focus on price elasticity. Pricing is relevant to virtually all companies.

under Data Science

Training is without a doubt the most important part of developing a machine learning application. It’s when you start realizing whether or not your model is worth it, how your hyperparameters should look like and what do you need to change in your architecture. In general, most machine learning engineers spend quite some time on training, experimenting with different models, tuning their architecture and discovering the best metrics and losses for their problem.

under Data Science

This book covers the building blocks of the most common methods in machine learning. This set of methods is like a toolbox for machine learning engineers. Those entering the field of machine learning should feel comfortable with this toolbox so they have the right tool for a variety of tasks. Each chapter in this book corresponds to a single machine learning method or group of methods. In other words, each chapter focuses on a single tool within the ML toolbox.

under Data Science

Our problem here is to define whether or not a certain news article is fake news. The dataset is comprised of 3997 news articles each includes a title, text, and the target label as a REAL/FAKE binary label. Part of the course was also testing the model on a test dataset but I never received target for this dataset. The accuracy score of cross validation testing within the training dataset was 94%.

under Data Science

It’s hard to imagine a modern, tech-literate business that doesn’t use data analysis, data science, machine learning, or artificial intelligence in some form. NumPy is at the core of all of those fields.

under Data Science

In this post, we are going to work with Pandas iloc, and loc. More specifically, we are going to learn slicing and indexing by iloc and loc examples. 

Once we have a dataset loaded as a Pandas dataframe, we often want to start accessing specific parts of the data based on some criteria. For instance, if our dataset contains the result of an experiment comparing different experimental groups, we may want to calculate descriptive statistics for each experimental group separately.

under Data Science

What is Pyjanitor? Before we continue learning on how to use Pandas and Pyjanitor to clean our datasets, we will learn about this package. The python package Pyjanitor extends Pandas with a verb-based API. This easy to use API is providing us with convenient data cleaning techniques. Apparently, it started out as a port of the R package janitor. Furthermore, it is inspired by the ease-of-use and expressiveness of the r-package dplyr. Note, there are some different ways how to work with the methods and this post will not cover all of them (see the documentation).

under Data Science

One of the most common mistakes data scientists make when training machine learning models is incorrectly splitting data for training and testing. The train/test split involves splitting data during the model training and evaluation process.

Learner makes this simple with a single parameter selection during the model building process. It’s also simple to set the percentage split between training and testing data for each model trained. 

under Data Science

This article introduces how to build a Python and Flask based web application for performing text analytics on internet resources such as blog pages. To perform text analytics I will utilizing Requests for fetching web pages, BeautifulSoup for parsing html and extracting the viewable text and, apply the TextBlob package to calculate a few sentiment scores.  The code for this article is hosted on GitHub so please fork and experiment with it.

under Data Science

Data pipelines are where most of the time is spent for those working with data because the bulk of a machine learning project involves data collection and cleaning. Loominus gives everyone the power to build the data pipelines critical to any machine learning project. 

Teraport is a powerful tool within the Loominus product suite that ingests and stages data. In another post, we’ll discuss the data ingestion APIs. For now we’ll focus on building a powerful data pipeline for feature engineering.

under Data Science

Hugging Face, the NLP startup behind several social AI apps and open source libraries such as PyTorch BERT, just released a new python library called PyTorch Transformers.

Transformers are a new set of techniques used to train highly performing and efficient models for performing natural language processing (NLP) and natural language understanding (NLU) tasks such as questions answering and sentiment analysis. Several of the recent techniques used to improve and advance the performance of NLP models, such as XLNet and BERT, are all based on a variation of Transformer.

under Data Science

Machine learning is pretty undeniably the hottest topic in data science right now. It’s also the basic concept that underpins some of the most exciting areas in technology, like self-driving cars and predictive analytics. Searches for Machine Learning on Google hit an all-time-high in April of 2019, and they interest hasn’t declined much since.

under Data Science

The pandas library is a powerful tool for multiple phases of the data science workflow, including data cleaning, visualization, and exploratory data analysis. However, the size and complexity of the pandas library makes it challenging to discover the best way to accomplish any given task.

under Data Science

The Pattern library is a multipurpose library capable of handling the following tasks:

  • Natural Language Processing: Performing tasks such as tokenization, stemming, POS tagging, sentiment analysis, etc.
  • Data Mining: It contains APIs to mine data from sites like Twitter, Facebook, Wikipedia, etc.
  • Machine Learning: Contains machine learning models such as SVM, KNN, and perceptron, which can be used for classification, regression, and clustering tasks.

In this article, we will see the first two applications of the Pattern library from the above list. We will explore the use of the Pattern Library for NLP by performing tasks such as tokenization, stemming and sentiment analysis. We will also see how the Pattern library can be used for web mining.

End-To-End Topic Modeling in Python towardsdatascience.com

under Data Science

Topic Model: In a nutshell, it is a type of statistical model used for tagging abstract “topics” that occur in a collection of documents that best represents the information in them.

Many techniques are used to obtain topic models. This post aims to demonstrate the implementation of LDA: a widely used topic modeling technique.

under Data Science

In this post, we’ll learn about Pandas, a high-performance open-source package for doing data analysis in Python.

We’ll cover:

  • What Pandas is and why should you use it.
  • What a Pandas DataFrame is.
  • Creating and viewing a DataFrame.
  • Manipulating data in a DataFrame.

Vaex: A DataFrame with super strings towardsdatascience.com

under Data Science

String manipulations are an essential part of Data Science. The latest release of Vaex adds incredibly fast and memory efficient support for all common string manipulations. Compared to Pandas, the most popular DataFrame library in the Python ecosystem, string operations are up to ~30–100x faster on your quadcore laptop, and up to a 1000 times faster on a 32 core machine.

under Data Science

In this article, we will explore TextBlob, which is another extremely powerful NLP library for Python. TextBlob is built upon NLTK and provides an easy to use interface to the NLTK library. We will see how TextBlob can be used to perform a variety of NLP tasks ranging from parts-of-speech tagging to sentiment analysis, and language translation to text classification.