GitHub – google-pytorch/torch_tpu · GitHub

This undertaking offers a PyTorch backend for Google’s Tensor Processing Units
(TPUs), enabling customers to run PyTorch fashions and operations on TPU gadgets. It
contains customized ATen kernels for varied PyTorch operations, a compilation cache
for optimizing execution, and examples for distributed coaching and mannequin
inference.

• PyTorch ATen kernel implementation for TPU.
• Compilation cache for optimized execution of XLA computations.
• Support for distributed coaching (Data Parallelism, Tensor Parallelism).
• Examples for varied fashions (Llama, Qwen, ResNet, DLRM, minGPT, UNet).
• Integration with torch.compile() for graph mode execution.
• Utilities for debugging and benchmarking.

A person information is accessible at
https://torchtpu.readthedocs.io/. This is a
static model of the interactive notebooks. They can be found within the docs
folder.

The notebooks are Marimo (much like Jupyter). They could be run with marimo edit file.py after which you possibly can entry them in your browser. You will must be
working this from a TPU VM, a v6e GCE occasion is best.

The identical directions can be found within the person information linked above.

TorchTPU’s Python model assist coverage matches
PyTorch’s Release Compatibility Matrix.
Currently, Python 3.11+ is supported. See the
Python Version Support Policy for
extra particulars.

First create a Python digital atmosphere:

python -m venv .venv
supply .venv/bin/activate

or, with uv

uv venv .venv
supply .venv/bin/activate

Install the most recent public launch
(0.2.0 on PyPI) (for uv prepend
uv to all pip instructions):

pip set up torch_tpu==0.2.0 --extra-index-url https://download.pytorch.org/whl/cpu

Wheels could be constructed through:

bazel construct --config=wheel_common //ci/wheel:torch_tpu_wheel

All OSS Bazel builds default to optimized mode (-c choose) mechanically through
.bazelrc. Unoptimized builds (-c fastbuild, -c dbg) can degrade
efficiency and should not used for wheel builds. If you’re creating or
debugging C++ code and explicitly need an unoptimized wheel, go
--//:allow_unoptimized_wheel=True:

bazel construct -c dbg --config=wheel_common //ci/wheel:torch_tpu_wheel 
    --//:allow_unoptimized_wheel=True

--config=wheel_common is required, and the construct fails at construct time with out
it. The config units --//:wheel_build=True, which routes the shared XLA/MLIR
backend by the torch_version-reset transition in order that pywrap elements it
right into a single libxla_base.so; constructed with out the flag, each per-version widespread
ships its personal copy of the backend and the ensuing wheel aborts on import torch with duplicate static registrations.

wheel_common additionally activates the Remote Build Execution (RBE) distant cache and
distant executor. If you shouldn’t have RBE credentials, append --config=no_rbe to
strip these flags again out:

bazel construct --config=wheel_common //ci/wheel:torch_tpu_wheel --config=no_rbe

The order issues right here: --config=no_rbe has to come back after
--config=wheel_common, as a result of the later config is the one which wins for the
remote-cache and remote-executor flags they each set.

Install wheel through:

cd <path_to_repo>
python -m venv .venv; supply .venv/bin/activate
# The extra-index-url offers the CPU model of torch moderately than the default CUDA construct.
pip set up ../bazel-bin/ci/wheel/*.whl --extra-index-url https://download.pytorch.org/whl/cpu

Note that putting in with out --extra-index-url https://download.pytorch.org/whl/cpu will set up the CUDA model of torch;
go --extra-index-url as proven above or set up the CPU construct individually.

Some tips that could get you began:

The undertaking makes use of Bazel for dependency administration. Key Python dependencies are:

TBD

This undertaking makes use of Bazel for constructing and dependency administration.

To set up Bazel, we really set up Bazelisk, which is a clear wrapper
that handles downloading and putting in Bazel itself.

There are a number of methods to put in Bazelisk. Here are some widespread methods:

See the
Bazelisk README
for extra set up directions

However you put in it, add it is location to PATH.

To confirm the set up, run bazel data within the repo’s root listing:

It ought to print details about the repo with out errors.

For C++ code navigation, it is strongly recommended to make use of VSCode with
clangd extension.
Relying on a
compilation database
generated from Bazel motion graph, clangd is ready to
enrich the editor with varied sensible options, together with code completions, go-to
definitions, and so on.

1. Install VSCode extensions through UI or the next command:

   code --install-extension llvm-vs-code-extensions.vscode-clangd
   # Uninstall Microsoft C++ extensions to keep away from interference.
   code --uninstall-extension ms-vscode.cpptools

2. Generate the preliminary compile_commands.json:

It might take some time (> half-hour) for the primary run to make a full construct, however
reruns will probably be incremental and low-cost. If you could have run bazel construct not too long ago,
generated information exist already so you possibly can go --no-build to skip ready for
heavy builds (observe that on a recent workspace, skipping the construct might stop
clangd from resolving generated information). See extra choices within the
script file-level feedback.

Alternatively, you possibly can invoke the script through nox (arguments after -- are
forwarded):

nox -s refresh_compile_commands -- --no-build
nox -s refresh_compile_commands -- //src/torch_tpu/ops/...

We welcome contributions to the PyTorch on TPU undertaking! Please see our
CONTRIBUTING.md for pointers on how you can contribute,
together with signing our Contributor License Agreement (CLA) and code overview
processes.

This undertaking is licensed below the Apache 2.0 License. See the
LICENSE file for particulars.