This repository has been archived on 2024-01-06. You can view files and clone it, but cannot push or open issues or pull requests.
justhomework/AIandML/e3_deep_learning/e3.0_tensor.ipynb

790 lines
18 KiB
Plaintext
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 实验3-1 Tensors\n",
"\n",
"实验目标:\n",
"\n",
"* 初步掌握PyTorch的张量用法\n",
"\n",
"张量(Tensor)是一种特殊的数据结构可简单理解为高维数组在使用方法上与数组或矩阵相似。在PyTorch中我们使用张量来描述模型的输入、输出以及模型参数。\n",
"\n",
"张量类似于NumPy的ndarrays并且张量可以在GPU上运行或其他专用硬件以实现加速计算。如果我们熟悉numpy.ndarray就很容易掌握PyTorch的Tensor。\n"
]
},
{
"cell_type": "code",
"execution_count": 62,
"metadata": {},
"outputs": [],
"source": [
"%matplotlib inline"
]
},
{
"cell_type": "code",
"execution_count": 63,
"metadata": {},
"outputs": [],
"source": [
"import torch\n",
"import numpy as np"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 1. Tensor 初始化\n",
"\n",
"张量可以通过多种方式初始化。请看以下示例:\n",
"\n",
"#### 1.1 直接来自数据\n",
"\n",
"张量可以直接从数据中创建。数据类型是自动推断的。"
]
},
{
"cell_type": "code",
"execution_count": 64,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([[1, 2],\n",
" [3, 4]])\n",
"torch.int64\n"
]
}
],
"source": [
"data = [[1, 2],[3, 4]]\n",
"x_data = torch.tensor(data)\n",
"print(x_data)\n",
"print(x_data.dtype)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### 1.2 从NumPy数组创建\n",
"\n",
"张量可以从NumPy数组创建反之亦然。"
]
},
{
"cell_type": "code",
"execution_count": 65,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"torch.int64\n"
]
}
],
"source": [
"np_array = np.array(data)\n",
"x_np = torch.from_numpy(np_array)\n",
"print(x_np.dtype)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### 1.3 来自另一个张量\n",
"\n",
"新张量保留参数张量的属性(形状、数据类型),除非显式重写。\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 66,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Ones Tensor: \n",
" tensor([[1, 1],\n",
" [1, 1]]) \n",
"\n",
"torch.int64\n",
"Random Tensor: \n",
" tensor([[0.4114, 0.9433],\n",
" [0.6890, 0.5708]]) \n",
"\n",
"torch.float32\n"
]
}
],
"source": [
"x_ones = torch.ones_like(x_data) # retains the properties of x_data\n",
"print(f\"Ones Tensor: \\n {x_ones} \\n\")\n",
"print(x_ones.dtype)\n",
"x_rand = torch.rand_like(x_data, dtype=torch.float) # overrides the datatype of x_data\n",
"print(f\"Random Tensor: \\n {x_rand} \\n\")\n",
"print(x_rand.dtype)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### 1.4 使用随机或常量值:\n",
"\n",
"''shape'' 是张量维数的元组。在下面的函数中,它确定输出张量的维数。\n"
]
},
{
"cell_type": "code",
"execution_count": 67,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"torch.float32\n",
"torch.float32\n",
"torch.float32\n",
"Random Tensor: \n",
" tensor([[0.3535, 0.5414, 0.5149],\n",
" [0.7026, 0.4758, 0.2522]]) \n",
"\n",
"Ones Tensor: \n",
" tensor([[1., 1., 1.],\n",
" [1., 1., 1.]]) \n",
"\n",
"Zeros Tensor: \n",
" tensor([[0., 0., 0.],\n",
" [0., 0., 0.]])\n"
]
}
],
"source": [
"shape = (2,3,)\n",
"rand_tensor = torch.rand(shape)\n",
"ones_tensor = torch.ones(shape)\n",
"zeros_tensor = torch.zeros(shape)\n",
"print(rand_tensor.dtype)\n",
"print(ones_tensor.dtype)\n",
"print(zeros_tensor.dtype)\n",
"\n",
"print(f\"Random Tensor: \\n {rand_tensor} \\n\")\n",
"print(f\"Ones Tensor: \\n {ones_tensor} \\n\")\n",
"print(f\"Zeros Tensor: \\n {zeros_tensor}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"> 请回答:\n",
"> 1. 采用以上各种方法创建Tensor时其数据类型即`.dtype`属性)是怎样的?"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"数据类型可在上面的输出中发现。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 2. Tensor 属性\n",
"\n",
"\n",
"张量属性描述它们的形状、数据类型以及存储它们的设备。\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 68,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Number of dimension: 2\n",
"Shape of tensor: torch.Size([5, 4])\n",
"Datatype of tensor: torch.float32\n",
"Device tensor is stored on: cpu\n"
]
}
],
"source": [
"tensor = torch.rand(5,4)\n",
"\n",
"print(f\"Number of dimension: {tensor.ndim}\")\n",
"print(f\"Shape of tensor: {tensor.shape}\")\n",
"print(f\"Datatype of tensor: {tensor.dtype}\")\n",
"print(f\"Device tensor is stored on: {tensor.device}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"> 请回答:\n",
"> 1. 修改变量 tensor 的尺寸,重新执行,给出结果。\n",
"> 2. 并根据结果分析和解释`tensor.ndim`、`tensor.shape`、`tensor.dtype`和`tensor.device`各是什么含义?\n",
"\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"分别为:维度、形状、数据类型、存储位置"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 3. Tensor 操作\n",
"\n",
"\n",
"* 在\"官网文档<https://pytorch.org/docs/stable/torch.html>\"中,详尽的介绍了约百余个张量的运算函数,包括转置、索引、切片、数学运算,线性代数,随机抽样等。\n",
"\n",
"* 需知晓的是这些函数都可以在GPU上运行。在批量化运行时GPU运算速度通常比在CPU上运行的速度更高。\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 69,
"metadata": {},
"outputs": [],
"source": [
"# We move our tensor to the GPU if available\n",
"if torch.cuda.is_available():\n",
" tensor = tensor.to('cuda')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"> 请回答:\n",
"> 1. 使用张量的任意操作函数,并给出代码和效果。\n",
"* 如果你熟悉NumPy API你会发现Tensor API使用起来轻而易举。\n",
"\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 70,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([[0.8814, 0.4445, 0.7783, 0.5680],\n",
" [0.8376, 0.7283, 0.5712, 0.6707],\n",
" [0.7175, 0.3082, 0.1837, 0.4545],\n",
" [0.2929, 0.8227, 0.3114, 0.1816],\n",
" [0.9362, 0.5050, 0.8165, 0.6510]], device='cuda:0')\n",
"tensor([[0.4919, 1.1102, 0.6788, 0.9667],\n",
" [0.5779, 0.7550, 0.9628, 0.8356],\n",
" [0.7705, 1.2575, 1.3861, 1.0990],\n",
" [1.2735, 0.6047, 1.2541, 1.3882],\n",
" [0.3591, 1.0414, 0.6154, 0.8619]], device='cuda:0')\n"
]
}
],
"source": [
"print(tensor)\n",
"print(torch.acos(tensor))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 4. 类似 numpy的索引(indexing)和切片(slicing) "
]
},
{
"cell_type": "code",
"execution_count": 71,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([[1., 0., 1., 1.],\n",
" [1., 0., 1., 1.],\n",
" [1., 0., 1., 1.],\n",
" [1., 0., 1., 1.]])\n"
]
}
],
"source": [
"tensor = torch.ones(4, 4)\n",
"tensor[:,1] = 0\n",
"print(tensor)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"> 请回答:\n",
"> 1. 请用索引或切片操作,给出`tensor`的第一行、第一列和中间2x2的子矩阵。"
]
},
{
"cell_type": "code",
"execution_count": 72,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([1., 0., 1., 1.])\n",
"tensor([1., 1., 1., 1.])\n",
"tensor([[0., 1.],\n",
" [0., 1.]])\n"
]
}
],
"source": [
"print(tensor[0])\n",
"print(tensor[:,0])\n",
"print(tensor[1:3,1:3])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 5. 合并Tensor\n",
"\n",
"合并Tensor有多种方式例如`torch.cat`和`torch.stack`。\n",
"\n",
"> 请回答:\n",
"> 1. 请查阅文档,并用示例描述`torch.cat`和`torch.stack`各是什么方式合并,有何不同。"
]
},
{
"cell_type": "code",
"execution_count": 73,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([[1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.],\n",
" [1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.],\n",
" [1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.],\n",
" [1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.]])\n"
]
}
],
"source": [
"t1 = torch.cat([tensor, tensor, tensor], dim=1)\n",
"print(t1)"
]
},
{
"cell_type": "code",
"execution_count": 74,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([[1., 2., 3.],\n",
" [1., 2., 3.]])\n",
"tensor([1., 2., 3., 1., 2., 3.])\n"
]
}
],
"source": [
" #注: .cat 和 .stack的区别在于 cat会增加现有维度的值,stack会新加增加一个维度\n",
"a=torch.Tensor([1,2,3])\n",
"print(torch.stack((a,a)))\n",
"print(torch.cat((a,a)))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 6. Tensors相乘\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 75,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor.mul(tensor) \n",
" tensor([[1., 0., 1., 1.],\n",
" [1., 0., 1., 1.],\n",
" [1., 0., 1., 1.],\n",
" [1., 0., 1., 1.]]) \n",
"\n",
"tensor * tensor \n",
" tensor([[1., 0., 1., 1.],\n",
" [1., 0., 1., 1.],\n",
" [1., 0., 1., 1.],\n",
" [1., 0., 1., 1.]])\n"
]
}
],
"source": [
"# This computes the element-wise product\n",
"print(f\"tensor.mul(tensor) \\n {tensor.mul(tensor)} \\n\")\n",
"# Alternative syntax:\n",
"print(f\"tensor * tensor \\n {tensor * tensor}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"这将计算两个张量之间的矩阵乘法\n"
]
},
{
"cell_type": "code",
"execution_count": 76,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor.matmul(tensor.T) \n",
" tensor([[3., 3., 3., 3.],\n",
" [3., 3., 3., 3.],\n",
" [3., 3., 3., 3.],\n",
" [3., 3., 3., 3.]]) \n",
"\n",
"tensor @ tensor.T \n",
" tensor([[3., 3., 3., 3.],\n",
" [3., 3., 3., 3.],\n",
" [3., 3., 3., 3.],\n",
" [3., 3., 3., 3.]])\n"
]
}
],
"source": [
"print(f\"tensor.matmul(tensor.T) \\n {tensor.matmul(tensor.T)} \\n\")\n",
"# Alternative syntax:\n",
"print(f\"tensor @ tensor.T \\n {tensor @ tensor.T}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"> 请回答:\n",
"> 1. 请查阅文档,并结合示例,说明以上两种乘法的区别"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"一种是按元素相乘,一种是矩阵乘法"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 7. 就地In-place操作\n",
"\n",
"> 就地操作是指操作的输入和输出都是同一个变量。例如C语言中的`x++`和`y*=5`都属于就地操作。由于就地操作避免了内存拷贝,可以提升运算速度。\n",
"\n",
"\n",
"具有`_`后缀的成员函数即为就地操作。例如:`x.copy_(y)`、`x.t_()`执行后将都将更改变量`x`。"
]
},
{
"cell_type": "code",
"execution_count": 77,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([[1., 0., 1., 1.],\n",
" [1., 0., 1., 1.],\n",
" [1., 0., 1., 1.],\n",
" [1., 0., 1., 1.]]) \n",
"\n",
"tensor([[6., 5., 6., 6.],\n",
" [6., 5., 6., 6.],\n",
" [6., 5., 6., 6.],\n",
" [6., 5., 6., 6.]])\n"
]
}
],
"source": [
"print(tensor, \"\\n\")\n",
"tensor.add_(5)\n",
"print(tensor)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"注意:\n",
"\n",
"就地操作虽然可节省一些内存,但在计算梯度时可能会出现问题。因为就地操作会丢失计算的历史。因此,不鼓励使用它们。\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"> 请回答:\n",
"> 1. 请尝试`x.copy_(y)`函数,并说明其用法。\n"
]
},
{
"cell_type": "code",
"execution_count": 78,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([[1, 2],\n",
" [3, 4]])\n",
"tensor([[4, 3],\n",
" [2, 1]])\n",
"tensor([[4, 3],\n",
" [2, 1]])\n"
]
}
],
"source": [
"#将y复制到x中\n",
"x = torch.tensor([[1, 2],[3, 4]])\n",
"print(x)\n",
"y = torch.tensor([[4, 3],[2,1]])\n",
"print(y)\n",
"x.copy_(y)\n",
"print(x)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 8. 与 NumPy 间的转换\n",
"\n",
"Torch的Tensor可以和NumPy的ndarray互相转换。\n",
"\n",
"若Tensor是在CPU上那么其转换所得 NumPy 数组可与之共享其底层内存。也就是,更改其中一个将导致另一个也被更改。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### 8.1 Tensor 转为 NumPy array\n",
"\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 79,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"t: tensor([1., 1., 1., 1., 1.])\n",
"n: [1. 1. 1. 1. 1.]\n"
]
}
],
"source": [
"t = torch.ones(5)\n",
"print(f\"t: {t}\")\n",
"n = t.numpy()\n",
"print(f\"n: {n}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"张量的更改亦反映在NumPy数组中。\n"
]
},
{
"cell_type": "code",
"execution_count": 80,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"t: tensor([2., 2., 2., 2., 2.])\n",
"n: [2. 2. 2. 2. 2.]\n"
]
}
],
"source": [
"t.add_(1)\n",
"print(f\"t: {t}\")\n",
"print(f\"n: {n}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### 8.2 NumPy array 转为 Tensor"
]
},
{
"cell_type": "code",
"execution_count": 81,
"metadata": {},
"outputs": [],
"source": [
"n = np.ones(5)\n",
"t = torch.from_numpy(n)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"更改NumPy数组亦影响张量。\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 82,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"t: tensor([2., 2., 2., 2., 2.], dtype=torch.float64)\n",
"n: [2. 2. 2. 2. 2.]\n"
]
}
],
"source": [
"np.add(n, 1, out=n)\n",
"print(f\"t: {t}\")\n",
"print(f\"n: {n}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"> 请回答:\n",
"> 1. 请查阅文档以了解如何复制Tensor或ndarray以避免内存共享时的干扰。请给出示例代码。"
]
},
{
"cell_type": "code",
"execution_count": 83,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([2., 2., 2.])\n",
"[2. 2. 2.]\n",
"tensor([2., 2., 2.])\n",
"[1. 1. 1.]\n"
]
}
],
"source": [
"#使用clone方法\n",
"a = torch.ones(3)\n",
"b = a.numpy()\n",
"a.add_(1)\n",
"print(a)\n",
"print(b) #a和b共享内存\n",
"\n",
"a = torch.ones(3)\n",
"b = a.clone().numpy()\n",
"a.add_(1)\n",
"print(a)\n",
"print(b) #a和b不共享内存"
]
}
],
"metadata": {
"kernelspec": {
"display_name": ".venv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.16"
},
"vscode": {
"interpreter": {
"hash": "0733c54d9044ea299f7b7f48049f3576c8ad4e6ff5a97e2c60d8a9e3bff0bc54"
}
}
},
"nbformat": 4,
"nbformat_minor": 1
}