Does the use of the bfloat16 data format require special programming techniques (Python) for TPU?
Tuesday, 16 September 2025
by Michał Otoka
The use of the bfloat16 (brain floating point 16) data format is a key consideration for maximizing performance and efficiency on Google Cloud TPUs, specifically with the TPU v2 and v3 architectures. Understanding whether its use requires special programming techniques in Python, especially when utilizing popular machine learning frameworks such as TensorFlow, is important for
How PyTorch reduces making use of multiple GPUs for neural network training to a simple and straightforward process?
Saturday, 02 September 2023
by EITCA Academy
PyTorch, an open-source machine learning library developed by Facebook’s AI Research lab, has been designed with a strong emphasis on flexibility and simplicity of use. One of the important aspects of modern deep learning is the ability to leverage multiple GPUs to accelerate neural network training. PyTorch was specifically designed to simplify this process in
- Published in Artificial Intelligence, EITC/AI/DLPP Deep Learning with Python and PyTorch, Advancing with deep learning, Computation on the GPU, Examination review
Tagged under:
Artificial Intelligence, DataParallel, DistributedDataParallel, Mixed Precision, Model Sharding, Multi-GPU, PyTorch