PyTorch provides a lot of methods for the Tensor type. Some of these methodsmay be confusing for new users. Here, I would like to talk about
view()
vsreshape()
,transpose()
vspermute()
.- Torch.view reshapes the tensor to a different but compatible shape. For example, our input tensor aten has the shape (2, 3).This can be viewed as tensors of shapes (6, 1), (1, 6) etc.
- Jul 15, 2020 Looping over the last dimension: 1.0844 seconds Permute and loop over the last dimension: 1.1350 seconds 1.0 factor increase Using the permute command has significantly reduced our looping time. There is, of course, a small overhead for using the permute function, but in almost all practical cases it is best to include it.
view() vs transpose()
Pytorch 1.4 cuda-toolkit 10.1 ubuntu 16.0.4. I am facing this problem only during backward computation in training. My evaluation code runs fine. Interestingly my code runs fine with this combination: pytorch1.3 cuda-toolkit 10.0 pytorch1.1 cuda-toolkit 9.0. Nov 27, 2020 bboxpred = bboxpred. Permute (0, 2, 3, 1). Reshape (-1, 4) while some carried-away examples might go to the extreme like. Non-Contiguous Tensors.
Both
view()
and reshape()
can be used to change the size or shape oftensors. But they are slightly different.The
view()
has existed for a long time. It will return a tensor with the newshape. The returned tensor shares the underling data with the original tensor.If you change the tensor value in the returned tensor, the corresponding valuein the viewed tensor also changes.On the other hand, it seems that
reshape()
has been introduced in version0.4. According to thedocument, thismethod willReturns a tensor with the same data and number of elements as input, but with the specified shape. When possible, the returned tensor will be a view of input. Otherwise, it will be a copy. Contiguous inputs and inputs with compatible strides can be reshaped without copying, but you should not depend on the copying vs. viewing behavior.
It means that
torch.reshape
may return a copy or a view of the originaltensor. You can not count on that to return a view or a copy. According to thedeveloper:if you need a copy use clone() if you need the same storage use view(). The semantics of reshape() are that it may or may not share the storage and you don’t know beforehand.
As a side note, I found that torch version 0.4.1 and 1.0.1 behaves differentlywhen you print the
id
of original tensor and viewing tensor:You see that
id
of a.storage()
and b.storage()
is not the same. Isn’tthat their underlying data the same? Why this difference?I filed an issue in thePyTorch repo and got answers from the developer. It turns out that to find thedata pointer, we have to use the
data_ptr()
method. You will find that theirdata pointers are the same.view() vs transpose()
transpose()
, like view()
can also be used to change the shape of a tensorand it also returns a new tensor sharing the data with the original tensor:Returns a tensor that is a transposed version of input. The given dimensions dim0 and dim1 are swapped.
The resulting out tensor shares it’s underlying storage with the input tensor, so changing the content of one would change the content of the other.
One difference is that
view()
can only operate on contiguous tensor and thereturned tensor is still contiguous. transpose()
can operate both oncontiguous and non-contiguous tensor. Unlike view()
, the returned tensor maybe not contiguous any more.2/3 Cup
But what does contiguous mean?
There is a good answer on SOwhich discusses the meaning of
contiguous
in Numpy. It also applies toPyTorch.Air radar 5 2 5 x 6. As I understand,
contiguous
in PyTorch means if the neighboring elements inthe tensor are actually next to each other in memory. Let’s take a simpleexample:Meta – music tag editor audio metadata 1 6 4. Tensor
x
and y
in the above example share the same memory space1.If you check their contiguity with
is_contiguous()
,you will find that x
is contiguous but y
is not.Since x is contiguous, x[0][0] and x[0][1] are next to each other in memory.But y[0][0] and y[0][1] is not.
A lot of tensor operations requires that the tensor should be contiguous,otherwise, an error will be thrown. To make a non-contiguous tensor becomecontiguous, use call the
contiguous()
,which will return a new contiguous tensor. In plain words, it will create a newmemory space for the new tensor and copy the value from the non-contiguoustensor to the new tensor.permute()
and tranpose()
are similar. transpose()
can only swap twodimension. But permute()
can swap all the dimensions. For example:Note that, in
permute()
, you must provide the new order of all thedimensions. Fortnite pc requirements amd. In transpose()
, you can only provide two dimensions. tranpose()
can be thought as a special case of permute()
method in for 2D tensors.2/3 As A Decimal
- tensor data pointers.
- view after transpose raises non-contiguous error.
- When to use which, permute, view, transpose.
- Difference between reshape() and view().
2/3 Fraction Symbol
- To show a tensor’s memory address, use
tensor.data_ptr()
. ↩︎