Resources for developers using Python for scientific computing and quantitative analysis
PyScript is a framework that allows users to create rich Python applications in the browser using HTML’s interface. PyScript aims to give users a first-class programming language that has consistent styling rules, is more expressive, and is easier to learn.
PostgresML is an end-to-end machine learning system. It enables you to train models and make online predictions using only SQL, without your data ever leaving your favorite database.
A time-series plot with a single line is a helpful graph to express data with long sequences. It consists of an X-axis representing the timeline and a Y-axis showing the value. This is a standard method since the concept is simple and easy to understand. The plot can help us extract some insight information such as trends and seasonal effects.
Turning to DuckDB when you need to crunch more numbers faster than pandas in your Streamlit app
In this video kaggle grandmaster Rob Mulla takes you through an economic data analysis project with python pandas. We walk through the process of pulling down the data for different economic indicators, cleaning and joining the data. Using the Fred api you can pull up to date data and compare, analyze and explore.
Datasets from real-world scenarios are important for building and testing machine learning models. You may just want to have some data to experiment with an algorithm. You may also want to evaluate your model by setting up a benchmark or determining its weaknesses using different sets of data. Sometimes, you may also want to create synthetic datasets, where you can test your algorithms under controlled conditions by adding noise, correlations, or redundant information to the data.
Recent trends in the modern web saw single page frameworks like React.js and Angular take over traditional multipage websites, mainly due to the lack of interactivity offered by HTML. It is worth noting however, that single page applications (SPAs) brought this interactivity at the cost of added complexity.
Blankly is a live trading engine, backtest runner and development framework wrapped into one powerful open source package. Models can be instantly backtested, paper traded, sandbox tested and run live by simply changing a single line. We built blankly for every type of quant including training & running ML models in the same environment, cross-exchange/cross-symbol arbitrage, and even long/short positions on stocks (all with built-in websockets).
In this blog, we’ll review how we took a raw .ipynb notebook that does time series forecasting with Arima, modularized it into a Ploomber pipeline, and ran parallel jobs on Slurm. You can follow the steps in this guide to deploy it yourself. We’ve been using this notebook by Willie Wheeler.
Our communications, both verbal and written, carry rich information. Even beyond what we are conveying explicitly, our tone, the selection of words add layers of meaning to the communication. As humans, we can understand these nuances, and often predict behavior using the information.
The amount of data available on the internet is quite staggering. It is often quite easy to do a quick search and click through to view data on a website. However, if you want to actually use that data in your analysis, you have to be able to fetch it and convert it into a format that is usable.
In this video Rob Mulla teaches how to make your pandas dataframes more efficient by casting dtypes correctly. This will make your code faster, use less memory and smaller when saving to disk or a database.
Enter your Alchemy API key and an NFT contract address, and with one command generate CSV extracts for all token transfers, historical sales on OpenSea, and each underlying item’s metadata (with calculated rarity scores).
You can read more from us about what this is, and why it matters, here.
Scaling up pandas is hard. With Modin, we took a first-principles approach to parallelizing the pandas API. Rather than focus on implementing what we knew was easy, we developed a theoretical basis for dataframes—the abstraction underlying pandas—and derived a dataframe algebra that can express the 600+ pandas operators in under 20 algebraic operators.
Stock returns can be heavily impacted by certain events. Sometimes these events are unexpected or a surprise (natural disasters, global pandemics, terrorism) and other times they are scheduled (presidential elections, earnings announcements, financial data releases). We can use pandas to obtain financial data and see the impacts of events the returns of stocks.
Pandas is a great tool for time series analysis of financial market data. Because pandas DataFrames and Series work well with a date/time based index, they can be used effectively to analyze historical data. By financial market data, I mean data like historical price information on a publicly traded financial instrument. However, any sort of historical financial information can be analyzed.
This paper discusses the contrast between overnight and intraday stock returns. In the paper, we learn that overnight stock returns far outpace returns seen intraday during regular trading hours. In other words, stocks move the most when markets are not open, but when trading is taking place, the net returns seem to be close to zero. The paper claims this is a conspiracy where large hedge funds are manipulating the market. In this article, we will try to recreate the basic results from the article and look at one part of overnight returns that the article doesn’t discuss.
Learn these fun, exciting, unusual and just plain awesome pandas tricks to solve problems from the Advent of Code.
In this project, we will use Python to generate a collection of unique profile-picture avatar by layering images from a directory. This is the technique used bymany popular NFT collections like Cryptopunks or Bored Ape.
tstock is a tool to easily generate stock charts from the command line.
With the proliferation of applications of Machine Learning (ML) and especially Deep Learning (DL) models in decision making, it is becoming more crucial to see through the black box and justify key business decisions based off the back of such models’ outputs. For example, if an ML model rejects a customer’s loan request or assigns a credit risk in peer-to-peer lending to a certain customer, giving business stakeholders an explanation about why this decision was made could be a powerful tool in encouraging the adaptation of the models. In many cases, interpretable ML is not just a business requirement but a regulatory requirement to understand why a certain decision or option was given to a customer. SHapley Additive exPlanations (SHAP) is an important tool one can leverage towards explainable AI and to help establish trust in the outcome of ML models and neural networks in solving business problems.
All the ways to slice.
Stock returns can be heavily impacted by certain events. Sometimes these events are unexpected or a surprise (natural disasters, global pandemics, terrorism) and other times they are scheduled (presidential elections, earnings announcements, financial data releases). We can use pandas to obtain financial data and see the impacts of events the returns of stocks.
Maybe you’re not a software architecture wonk aficionado. That’s fair: I’d speculate that’s the exception rather than the rule for folks using pandas. This is not a deep dive into how the extension mechanisms are implemented. Interesting in their own right but out of the intended scope.
By the end of this article, you will be able to write an information extraction NLP pipeline using spaCy’s Matcher. It will extract dividend information from news headlines and articles.
You should be interested in this article if you like algorithmic trading and stock screening using Python. Mining unstructured text data for insights is what could set your algorithm apart.
GS Quant is a Python toolkit for quantitative finance, created on top of one of the world’s most powerful risk transfer platforms. Designed to accelerate development of quantitative trading strategies and risk management solutions, crafted over 25 years of experience navigating global markets. Designed and used by the quants at Goldman Sachs.
The Python scientific visualisation landscape is huge. It is composed of a myriad of tools, ranging from the most versatile and widely used down to the more specialised and confidential. Some of these tools are community based while others are developed by companies. Some are made specifically for the web, others are for the desktop only, some deal with 3D and large data, while others target flawless 2D rendering
Pandas is a great tool for time series analysis of financial market data. Because pandas DataFrames and Series work well with a date/time based index, they can be used effectively to analyze historical data. By financial market data, I mean data like historical price information on a publicly traded financial instrument. However, any sort of historical financial information can be analyzed.
In this lecture, we review the theory behind Metropolis Monte Carlo modeling and apply these concepts to the simulations of atomic systems.
The Shapley value is a method used in game theory that involves fairly distributing both gains and costs to actors working in a coalition.
Since each actor contributes differently to the coalition, the Shapley value makes sure that each actor gets a fair share depending on how much they contribute.
Blankly is an elegant python library for interacting with crypto, stocks, and forex for in a consistent and streamlined way. Now, no more reading API or struggling to get data. Blankly offers a powerful feature-set, optimized for speed and ease of use, better backtesting, and ultimately better models.
We’re bridging the gap between local development systems & live APIs by building a framework which allows backtesting, paper trading, sandbox testing, and live cross-exchange deployment without modifying a single line of trading logic.
This tutorial demonstrates how to apply clustering algorithms with Python to a dataset with two concrete use cases. The first example uses clustering to identify meaningful groups of Greco-Roman authors based on their publications and their reception. The second use case applies clustering algorithms to textual data in order to discover thematic groups. After finishing this tutorial, you will be able to use clustering in Python with Scikit-learn applied to your own data, adding an invaluable method to your toolbox for exploratory data analysis.
In this tutorial, we will create an Ethereum token on the Polygon Network from scratch. To create our token we will use Python and Python-like programming languages (Brownie and Vyper, we will learn more about them later).
By the end of this tutorial, you will have a personal token on a real Polygon network and hopefully a better understanding of how everything works on the Ethereum network.
The Alpaca brokerage service is very useful for algorithmic traders that comes with an API to retrieve data and execute trades in a paper or live environment. While you can also check the status and returns of your positions through the API, Alpaca has spent some time creating a frontend where users can visually check their live and paper accounts. Seeing that Alpaca is more focused on building out robust API’s for its users, the frontend is very simple and provides the bare necessities.
In this tutorial, we will look at how we can speed up scientific computations using multiprocessing in a real-world example. Specifically, we will detect the location of all nuclei within fluorescence microscopy images from the public MCF7 Cell Painting dataset released by the Broad Institute.
Reddit has been at the epicenter of one of the biggest movements in the world of finance, and although it seemed like an unlikely source of such a movement — it’s hardly surprising in hindsight.
The trading-focused subreddits of Reddit are the backdrop for a huge amount of discussion about what is happening in the markets — so it is only logical to tap into this huge data source.
When building a data extraction tool like this, one of the first things we need to do is identify what the data we’re extracting is actually about — and for that we will be using named entity recognition (NER).
As a private investor, the sheer amount of information that can be found on the internet is rather daunting. Trying to understand what type of companies or ETFs are available is incredibly challenging with there being millions of companies amd derivatives available on the market. Sure, the most traded companies and ETFs can quickly be found simply because they are known to the public (for example, Microsoft, Tesla, S&P500 ETF or an All-World ETF). However, what else is out there is often unknown.
In this article the author uses Reddit sentiment data to inform trading strategies. He derives market sentiment in two ways using the wallstreetbets subreddit:
For the most part, this book follows the standard material taught at the University of California, Berkeley, in the class E7: Introduction to computer programming for scientists and engineers. This class is taken by most science and engineering freshmen in the College of Engineering, and by undergraduate students from other disciplines, including physics, biology, Earth, and cognitive sciences. The course was originally taught in Matlab, but with the recent trend of the data science movement at Berkeley, the Division of Data Sciences agreed on and supported the transform of this course into a Pythonoriented course to prepare students from different fields for further data science courses.
When trading in markets such as equities or currencies it is important to identify value areas to inform our trading decisions. One way to do this is by looking at the volume profile.
In this post, we explore quantitative methods for examining the distribution of volume over a period of time.
More specifically, we’ll be using Python and statistical and signal processing tools in SciPy’s suite of modules. Data plots are rendered with Plotly.
This library provides high-performance components leveraging the hardware acceleration support and automatic differentiation of TensorFlow. The library will provide TensorFlow support for foundational mathematical methods, mid-level methods, and specific pricing models. The coverage is being expanded over the next few months.
The library is structured along three tiers:
We aim for the library components to be easily accessible at each level. Each layer will be accompanied by many examples which can be run independently of higher level components.
One of Excel’s benefits is that it offers an intuitive and powerful graphical interface for viewing your data. In contrast, pandas + a Jupyter notebook offers a lot of programmatic power but limited abilities to graphically display and manipulate a DataFrame view.
There are several tools in the Python ecosystem that are designed to fill this gap. They range in complexity from simple JavaScript libraries to complex, full-featured data analysis engines. The one common denominator is that they all provide a way to view and selectively filter your data in a graphical format. From this point of commonality they diverge quite a bit in design and functionality.
This article will review several of these DataFrame visualization options in order to give you an idea of the landscape and evaluate which ones might be useful for your analysis process.
In this project, we are going to build a python script that will keep track of the latest bitcoin price. And it will send you a telegram message every 30 minutes (you can tweak that) with the latest 6 bitcoin prices (again you can tweak that too). You can set a minimum threshold value so that if the BTC price goes below that threshold, then the script will send an immediate alert message showing the price.
End-to-end project: get the data, train the model, place the order, get notified.
NumPy is a fundamental library that most of the widely used Python data processing libraries are built upon (pandas, OpenCV), inspired by (PyTorch), or can efficiently share data with (TensorFlow, Keras, etc). Understanding how NumPy works gives a boost to your skills in those libraries as well. It is also possible to run NumPy code with no or minimal changes on GPU.
Learn how to perform algorithmic trading using Python in this complete course. Algorithmic trading means using computers to make investment decisions. Computer algorithms can make trades at a speed and frequency that is not possible by a human.
Mark 27.1 of the NAG Library contains a new routine, s30acf, for computing the implied volatility of a European option contract for arrays of input data.
This routine gives the user a choice of two algorithms. The first is the method of Jäckel (2015), which uses a third order Householder method to achieve close to machine accuracy for all but the most extreme inputs. This method is fast for short vectors of input data.
The second algorithm is based on that of Glau et al. (2018), with additional performance enhancements developed in a collaboration between NAG and mathematicians at Queen Mary University of London. This method uses Chebyshev interpolation and is designed for long vectors of input data, where vector instructions can be exploited. For applications in which accuracy to machine precision is not required, the algorithm can also be instructed to aim for accuracy to roughly single precision (approximately seven decimal places), giving even further performance improvements.
A complete, step by step guide to building a production-grade machine learning app with Django, PostgreSQL, React, Redux and Docker
