iOS Metal: ‘memory_format’ argument is incompatible with Metal tensor

Question:
Hello!
I am trying to use a pytorch model with Metal on iOS, but I keep getting these errors:
2021-12-04 23:40:04.794982+0100 ObjectDetection[1438:328965] 'memory_format' argument is incompatible with Metal tensor

  Debug info for handle(s): -1, was not found.

Exception raised from empty at /Users/distiller/project/aten/src/ATen/native/metal/MetalAten.mm:84 (most recent call first):
frame #0: _ZN3c106detail14torchCheckFailEPKcS2_jS2_ + 92 (0x104f03b04 in ObjectDetection)
frame #1: _ZN2at6native5metal5emptyEN3c108ArrayRefIxEENS2_8optionalINS2_10ScalarTypeEEENS5_INS2_6LayoutEEENS5_INS2_6DeviceEEENS5_IbEENS5_INS2_12MemoryFormatEEE + 284 (0x104d1ca5c in ObjectDetection)
frame #2: _ZN2at12_GLOBAL__N_119empty_memory_formatEN3c108ArrayRefIxEENS1_8optionalINS1_10ScalarTypeEEENS4_INS1_6LayoutEEENS4_INS1_6DeviceEEENS4_IbEENS4_INS1_12MemoryFormatEEE + 220 (0x10428e2fc in ObjectDetection)
frame #3: _ZNK3c1010Dispatcher4callIN2at6TensorEJNS_8ArrayRefIxEENS_8optionalINS_10ScalarTypeEEENS6_INS_6LayoutEEENS6_INS_6DeviceEEENS6_IbEENS6_INS_12MemoryFormatEEEEEET_RKNS_19TypedOperatorHandleIFSG_DpT0_EEESJ_ + 220 (0x10410b9dc in ObjectDetection)
frame #4: _ZN2at4_ops19empty_memory_format4callEN3c108ArrayRefIxEENS2_8optionalINS2_10ScalarTypeEEENS5_INS2_6LayoutEEENS5_INS2_6DeviceEEENS5_IbEENS5_INS2_12MemoryFormatEEE + 144 (0x1040d00d0 in ObjectDetection)
frame #5: _ZN2at12_GLOBAL__N_141structured_elu_default_backend_functional10set_outputExN3c108ArrayRefIxEES4_NS2_13TensorOptionsENS3_INS_7DimnameEEE + 152 (0x1043d2968 in ObjectDetection)
frame #6: _ZN2at18TensorIteratorBase11fast_set_upERKNS_20TensorIteratorConfigE + 268 (0x104d1a424 in ObjectDetection)
frame #7: _ZN2at18TensorIteratorBase5buildERNS_20TensorIteratorConfigE + 152 (0x104d17d38 in ObjectDetection)
frame #8: _ZN2at18TensorIteratorBase14build_unary_opERKNS_10TensorBaseES3_ + 152 (0x104d187cc in ObjectDetection)
frame #9: _ZN2at12_GLOBAL__N_111wrapper_eluERKNS_6TensorERKN3c106ScalarES7_S7_ + 136 (0x10437d744 in ObjectDetection)
frame #10: _ZN3c104impl34call_functor_with_args_from_stack_INS0_6detail31WrapFunctionIntoRuntimeFunctor_IPFN2at6TensorERKS5_RKNS_6ScalarESA_SA_ES5_NS_4guts8typelist8typelistIJS7_SA_SA_SA_EEEEELb0EJLm0ELm1ELm2ELm3EEJS7_SA_SA_SA_EEENSt3__15decayINSD_21infer_function_traitsIT_E4type11return_typeEE4typeEPNS_14OperatorKernelENS_14DispatchKeySetEPNSI_6vectorINS_6IValueENSI_9allocatorISU_EEEENSI_16integer_sequenceImJXspT1_EEEEPNSF_IJDpT2_EEE + 140 (0x10432c65c in ObjectDetection)
frame #11: _ZN3c104impl31make_boxed_from_unboxed_functorINS0_6detail31WrapFunctionIntoRuntimeFunctor_IPFN2at6TensorERKS5_RKNS_6ScalarESA_SA_ES5_NS_4guts8typelist8typelistIJS7_SA_SA_SA_EEEEELb0EE4callEPNS_14OperatorKernelERKNS_14OperatorHandleENS_14DispatchKeySetEPNSt3__16vectorINS_6IValueENSP_9allocatorISR_EEEE + 40 (0x10432c56c in ObjectDetection)
frame #12: _ZNK3c1010Dispatcher9callBoxedERKNS_14OperatorHandleEPNSt3__16vectorINS_6IValueENS4_9allocatorIS6_EEEE + 128 (0x104deb470 in ObjectDetection)
frame #13: _ZNSt3__110__function6__funcIZN5torch3jit6mobile8Function15append_operatorERKNS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEESD_RKN3c108optionalIiEExE3$_3NS9_ISJ_EEFvRNS_6vectorINSE_6IValueENS9_ISM_EEEEEEclESP_ + 668 (0x104debfa8 in ObjectDetection)
frame #14: _ZN5torch3jit6mobile16InterpreterState3runERNSt3__16vectorIN3c106IValueENS3_9allocatorIS6_EEEE + 4200 (0x104df6134 in ObjectDetection)
frame #15: _ZNK5torch3jit6mobile8Function3runERNSt3__16vectorIN3c106IValueENS3_9allocatorIS6_EEEE + 192 (0x104dea584 in ObjectDetection)
frame #16: _ZNK5torch3jit6mobile6Method3runERNSt3__16vectorIN3c106IValueENS3_9allocatorIS6_EEEE + 544 (0x104dfcf50 in ObjectDetection)
frame #17: _ZNK5torch3jit6mobile6MethodclENSt3__16vectorIN3c106IValueENS3_9allocatorIS6_EEEE + 24 (0x104dfdad0 in ObjectDetection)
frame #18: _ZN5torch3jit6mobile6Module7forwardENSt3__16vectorIN3c106IValueENS3_9allocatorIS6_EEEE + 148 (0x104eb23a8 in ObjectDetection)
frame #19: -[InferenceModule + + (0x104eb1b20 in ObjectDetection)
frame #20: $s15ObjectDetection14ViewControllerC9runTappedyyypFyycfU_ + 1072 (0x104ecc234 in ObjectDetection)
frame #21: $sIeg_IeyB_TR + 48 (0x104ebb15c in ObjectDetection)
frame #22: _dispatch_call_block_and_release + 24 (0x1068e7ce4 in libdispatch.dylib)
frame #23: _dispatch_client_callout + 16 (0x1068e9528 in libdispatch.dylib)
frame #24: _dispatch_queue_override_invoke + 888 (0x1068ebcc4 in libdispatch.dylib)
frame #25: _dispatch_root_queue_drain + 376 (0x1068fb048 in libdispatch.dylib)
frame #26: _dispatch_worker_thread2 + 152 (0x1068fb970 in libdispatch.dylib)
frame #27: _pthread_wqthread + 212 (0x1d24be568 in libsystem_pthread.dylib)
frame #28: start_wqthread + 8 (0x1d24c1874 in libsystem_pthread.dylib)

I searched for answers, and found this issue 1. The solution was that the Pytorch backend doesn’t support quantized models. However, I think that’s not the problem in my case. I use this model, and I could not find any signs of quantization. Furthermore, in this paper, the creator of the model ends the conclusion with these words:

Some techniques may further improve performance and accuracy, such as quantization, pruning, knowledge distillation. We left them for the future research.

I also print out the is_quantized property every time the output is returned, and it’s false every single time.
The forward function:
    def forward(self, x):
        backbone_features = self.model(x)
        backbone_features = self.cpm(backbone_features)

        stages_output = self.initial_stage(backbone_features)
        for refinement_stage in self.refinement_stages:
            stages_output.extend(
                refinement_stage(torch.cat([backbone_features, stages_output[-2], stages_output[-1]], dim=1)))

        for i, tensor in enumerate(stages_output):
            print(f'output {i} shape = {tensor.shape}')
            print(f'output {i} is_quantized: {tensor.is_quantized}')
        print("--------------------next frame -----------------------")

        return stages_output

The output of the prints:

Képernyőfotó 2021-12-05 - 0.15.411658×742 191 KB

Inference code on the iOS side:
- (NSArray<NSNumber*>*)detectImage:(void*)imageBuffer {
    try {

        at::Tensor tensor = torch::from_blob(imageBuffer, { 1, 3, input_height, input_width }, at::kFloat).metal();

        c10::InferenceMode guard;
        CFTimeInterval startTime = CACurrentMediaTime();
        auto outputTensorList = _impl.forward({ tensor }).toTensorList();
        CFTimeInterval elapsedTime = CACurrentMediaTime() - startTime;
        NSLog(@"inference time:%f", elapsedTime);

        auto heatmap1 = outputTensorList.get(0).dequantize().cpu();

        // outputs 19 pieces of 80*80 matrices

        auto heatmap_pixels = heatmap1[0][18]; // the 19th heatmap, 80*80
        auto dim1 = heatmap_pixels.size(0);
        auto dim2 = heatmap_pixels.size(1);

        float* floatBuffer = heatmap_pixels.data_ptr<float>();
        if (!floatBuffer) {
            return nil;
        }

        NSMutableArray* results = [[NSMutableArray alloc] init];
        for (int i = 0; i < (dim1*dim2); i++) {
          [results addObject:@(floatBuffer[i])];
        }
        return [results copy];

    } catch (const std::exception& exception) {
        NSLog(@"%s", exception.what());
    }
    return nil;
}

The model saving code:
    net = PoseEstimationWithMobileNet()
    checkpoint = torch.load(args.checkpoint_path, map_location='cpu')
    load_state(net, checkpoint)

    torch.save(net.state_dict(), '/Users/morvaybalazs/PycharmProjects/model.pt')
    model = PoseEstimationWithMobileNet()
    model.load_state_dict(torch.load('/Users/morvaybalazs/PycharmProjects/model.pt'))
    model.eval()
    traced_script_module = torch.jit.script(model)
    from torch.utils.mobile_optimizer import optimize_for_mobile
    torchscript_model_optimized = optimize_for_mobile(traced_script_module, backend='metal')
    print(torch.jit.export_opnames(torchscript_model_optimized))
    torchscript_model_optimized._save_for_lite_interpreter("mobile_model_metal.pt")

The exception is thrown when the forward function gets called.
Is it possible, that the model still uses quantization?
I am a beginner in Python and Pytorch, so any help or comment is much appreciated!

Context:
I had a similar problem with my model. Try to experiment with the input tensor you use in your Swift code. In my case I replaced the input tensor with a random one and then it worked.
at::Tensor tensor = torch::rand({ 1, 3, input_height, input_width }, at::kFloat).metal();

I don’t know how to go on from there, but maybe this leads you on the right track. Let me know if you find out more.


Mobile optimized model raised error “required rank 4 tensor to use channels_last format”

Question:
I am trying to export a custom model (VGG + LSTM) from pytorch to an Android phone. I am using pytorch 1.9 both on the laptop and the android device.
The exported model is optimized with this:
traced_model = torch.jit.trace(rnn_model, example_inputs=x_dummy)
traced_model = optimize_for_mobile(traced_model)

But when I run model.forward(…) on the android device I get the following error:
Pending exception com.facebook.jni.CppException: required rank 4 tensor to use channels_last format

Everything works fine without optimization. Any idea?

Context: Have you tested the optimized model in Python? @kimishpatel any idea?


Question: Is there pytorch.js library (similar to tensorflow.js) to train models  directlly on mobile edevices?

Context: Hello Rohit! Unfortunately there’s isn’t a JS library for on-device training available at this point in time. However, you could train models directly using the C++ API since the full-jit interpreter can run on mobile platforms.


Question:
Hello @all
i am facing the issue in converting dictionary to tenor in iOS

Context: Hi @patel_shahrukh, could you elaborate more about your use case, do you want to convert iOS Dictionary(NSDictionary) to PyTorch Tensor, or IValue? What issue are you facing?


aten/src/ATen/Tensor.h:3:10: fatal error: ‘ATen/core/TensorBody.h’ file not found

Question:
Hello,
I’m trying to build pytorch v1.10 and get the speed_benchmark_torch binary described in Pytorch Mobile Performance Recipes — PyTorch Tutorials 1.10.0+cu102 documentation 2. Using that command:
BUILD_PYTORCH_MOBILE=1 ANDROID_ABI=arm64-v8a ./scripts/build_android.sh -DBUILD_BINARY=ON

(I am using AndroidNDK-r21, with set ANDROID_NDK)
I am getting this error
aten/src/ATen/Tensor.h:3:10: fatal error: 'ATen/core/TensorBody.h' file not found

(I’ve seen Build libtorch for android error, ‘ATen/core/TensorBody.h’ file not found 1, but it is using a different build command. I tried
BUILD_PYTORCH_MOBILE=1 ANDROID_ABI=arm64-v8a ./scripts/build_pytorch_android.sh -DBUILD_BINARY=ON

but it doesn’t seem to support building the binary.)
Tried uploading the full log, but it said only image attachments are supported?!
So here is some of the log:
--
-- ******** Summary ********
-- General:
--   CMake version         : 3.16.3
--   CMake command         : /usr/bin/cmake
--   System                : Android
--   C++ compiler          : /home/sam/CK-SDKs/AndroidNDK-r21/android-ndk-r21e/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++
--   C++ compiler id       : Clang
--   C++ compiler version  : 9.0
--   Using ccache if found : ON
--   Found ccache          : CCACHE_PROGRAM-NOTFOUND
--   CXX flags             : -g -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -frtti -fexceptions  -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DUSE_VULKAN_WRAPPER -DUSE_KINETO -DLIBKINETO_NOCUPTI -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DUSE_VULKAN -DUSE_VULKAN_API -DBUILD_LITE_INTERPRETER -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -Wno-invalid-partial-specialization -Wno-typedef-redefinition -Wno-unknown-warning-option -Wno-unused-private-field -Wno-inconsistent-missing-override -Wno-aligned-allocation-unavailable -Wno-c++14-extensions -Wno-constexpr-not-const -Wno-missing-braces -Qunused-arguments -fcolor-diagnostics -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -g0
--   Build type            : Release
--   Compile definitions   :
--   CMAKE_PREFIX_PATH     : /home/sam/envs/py8/lib/python3.8/site-packages;/home/sam/CK-SDKs/AndroidNDK-r21/android-ndk-r21e/toolchains/llvm/prebuilt/linux-x86_64
--   CMAKE_INSTALL_PREFIX  : /home/sam/tmp/pytorch-v1.10.0/build_android/install
--   USE_GOLD_LINKER       : OFF
--
--   TORCH_VERSION         : 1.10.0
--   CAFFE2_VERSION        : 1.10.0
--   BUILD_CAFFE2          : ON
--   BUILD_CAFFE2_OPS      : OFF
--   BUILD_CAFFE2_MOBILE   : OFF
--   BUILD_STATIC_RUNTIME_BENCHMARK: OFF
--   BUILD_TENSOREXPR_BENCHMARK: OFF
--   BUILD_BINARY          : ON
--   BUILD_CUSTOM_PROTOBUF : OFF
--     Protobuf compiler   :
--     Protobuf includes   :
--     Protobuf libraries  :
--   BUILD_DOCS            : OFF
--   BUILD_PYTHON          : OFF
--   BUILD_SHARED_LIBS     : OFF
--   CAFFE2_USE_MSVC_STATIC_RUNTIME     : ON
--   BUILD_TEST            : OFF
--   BUILD_JNI             : OFF
--   BUILD_MOBILE_AUTOGRAD : OFF
--   BUILD_LITE_INTERPRETER: ON
--   INTERN_BUILD_MOBILE   : ON
--   USE_BLAS              : 1
--     BLAS                :
--   USE_LAPACK            : 0
--   USE_ASAN              : OFF
--   USE_CPP_CODE_COVERAGE : OFF
--   USE_CUDA              : OFF
--   USE_ROCM              : OFF
--   USE_EIGEN_FOR_BLAS    : ON
--   USE_FBGEMM            : OFF
--     USE_FAKELOWP          : OFF
--   USE_KINETO            : ON
--   USE_FFMPEG            : OFF
--   USE_GFLAGS            : OFF
--   USE_GLOG              : OFF
--   USE_LEVELDB           : OFF
--   USE_LITE_PROTO        : OFF
--   USE_LMDB              : OFF
--   USE_METAL             : OFF
--   USE_PYTORCH_METAL     : OFF
--   USE_PYTORCH_METAL_EXPORT     : OFF
--   USE_FFTW              : OFF
--   USE_MKL               :
--   USE_MKLDNN            : OFF
--   USE_NCCL              : OFF
--   USE_NNPACK            : ON
--   USE_NUMPY             : ON
--   USE_OBSERVERS         : OFF
--   USE_OPENCL            : OFF
--   USE_OPENCV            : OFF
--   USE_OPENMP            : OFF
--   USE_TBB               : OFF
--   USE_VULKAN            : ON
--     USE_VULKAN_FP16_INFERENCE    : OFF
--     USE_VULKAN_RELAXED_PRECISION : OFF
--     USE_VULKAN_SHADERC_RUNTIME   : OFF
--   USE_PROF              : OFF
--   USE_QNNPACK           : OFF
--   USE_PYTORCH_QNNPACK   : ON
--   USE_REDIS             : OFF
--   USE_ROCKSDB           : OFF
--   USE_ZMQ               : OFF
--   USE_DISTRIBUTED       : OFF
--   USE_DEPLOY           : OFF
--   USE_BREAKPAD         : ON
--   Public Dependencies  : Threads::Threads
--   Private Dependencies : eigen_blas;pthreadpool;cpuinfo;pytorch_qnnpack;nnpack;XNNPACK;VulkanWrapper;fp16;log;fmt::fmt-header-only;kineto;dl
--   USE_COREML_DELEGATE     : OFF
-- Configuring done
-- Generating done
-- Build files have been written to: /home/sam/tmp/pytorch-v1.10.0/build_android
Will install headers and libs to /home/sam/tmp/pytorch-v1.10.0/build_android/install for further Android project usage.
Scanning dependencies of target clog
Scanning dependencies of target pthreadpool
Scanning dependencies of target fmt
Scanning dependencies of target VulkanWrapper
[  1%] Building C object confu-deps/cpuinfo/deps/clog/CMakeFiles/clog.dir/src/clog.c.o
[  1%] Building C object confu-deps/pthreadpool/CMakeFiles/pthreadpool.dir/src/legacy-api.c.o
[  1%] Building CXX object third_party/fmt/CMakeFiles/fmt.dir/src/format.cc.o
[  1%] Building CXX object CMakeFiles/VulkanWrapper.dir/home/sam/CK-SDKs/AndroidNDK-r21/android-ndk-r21e/sources/third_party/vulkan/src/common/vulkan_wrapper.cpp.o
Scanning dependencies of target eigen_blas
[  1%] Building CXX object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/single.cpp.o
[  1%] Linking C static library ../../../../lib/libclog.a
[  1%] Built target clog
[  1%] Linking CXX static library lib/libVulkanWrapper.a
Scanning dependencies of target breakpad_common
[  1%] Building CXX object third_party/breakpad/CMakeFiles/breakpad_common.dir/src/common/convert_UTF.cc.o
[  1%] Building C object confu-deps/pthreadpool/CMakeFiles/pthreadpool.dir/src/portable-api.c.o
[  1%] Built target VulkanWrapper
[  1%] Building CXX object third_party/breakpad/CMakeFiles/breakpad_common.dir/src/common/linux/file_id.cc.o
Scanning dependencies of target libkineto_defs.bzl
[  1%] Built target libkineto_defs.bzl
[  1%] Building CXX object third_party/breakpad/CMakeFiles/breakpad_common.dir/src/common/linux/guid_creator.cc.o
Scanning dependencies of target kineto_api
[  1%] Building CXX object third_party/kineto/libkineto/CMakeFiles/kineto_api.dir/src/ThreadUtil.cpp.o
[  1%] Building CXX object third_party/breakpad/CMakeFiles/breakpad_common.dir/src/common/linux/memory_mapped_file.cc.o
[  1%] Building C object confu-deps/pthreadpool/CMakeFiles/pthreadpool.dir/src/memory.c.o
[  1%] Building CXX object third_party/breakpad/CMakeFiles/breakpad_common.dir/src/common/linux/linux_libc_support.cc.o
[  1%] Building C object confu-deps/pthreadpool/CMakeFiles/pthreadpool.dir/src/pthreads.c.o
[  1%] Building C object third_party/breakpad/CMakeFiles/breakpad_common.dir/src/common/linux/breakpad_getcontext.S.o
[  1%] Building CXX object third_party/breakpad/CMakeFiles/breakpad_common.dir/src/common/linux/elfutils.cc.o
[  1%] Linking C static library ../../lib/libpthreadpool.a
[  1%] Built target pthreadpool
Scanning dependencies of target c10
[  1%] Building CXX object c10/CMakeFiles/c10.dir/core/Allocator.cpp.o
[  1%] Building CXX object third_party/breakpad/CMakeFiles/breakpad_common.dir/src/common/linux/safe_readlink.cc.o
[  1%] Building CXX object third_party/breakpad/CMakeFiles/breakpad_common.dir/src/common/string_conversion.cc.o
[  1%] Building CXX object c10/CMakeFiles/c10.dir/core/AutogradState.cpp.o
[  1%] Linking CXX static library ../../lib/libbreakpad_common.a
[  1%] Built target breakpad_common
[  1%] Building CXX object c10/CMakeFiles/c10.dir/core/CPUAllocator.cpp.o
Scanning dependencies of target ATEN_CPU_FILES_GEN_TARGET
[  1%] Generating ../aten/src/ATen/CPUFunctions.h, ../aten/src/ATen/CPUFunctions_inl.h, ../aten/src/ATen/CompositeExplicitAutogradFunctions.h, ../aten/src/ATen/CompositeExplicitAutogradFunctions_inl.h, ../aten/src/ATen/CompositeImplicitAutogradFunctions.h, ../aten/src/ATen/CompositeImplicitAutogradFunctions_inl.h, ../aten/src/ATen/Declarations.yaml, ../aten/src/ATen/Functions.cpp, ../aten/src/ATen/Functions.h, ../aten/src/ATen/NativeFunctions.h, ../aten/src/ATen/NativeMetaFunctions.h, ../aten/src/ATen/Operators.h, ../aten/src/ATen/Operators_0.cpp, ../aten/src/ATen/Operators_1.cpp, ../aten/src/ATen/Operators_2.cpp, ../aten/src/ATen/Operators_3.cpp, ../aten/src/ATen/Operators_4.cpp, ../aten/src/ATen/RedispatchFunctions.h, ../aten/src/ATen/RegisterBackendSelect.cpp, ../aten/src/ATen/RegisterCPU.cpp, ../aten/src/ATen/RegisterCompositeExplicitAutograd.cpp, ../aten/src/ATen/RegisterCompositeImplicitAutograd.cpp, ../aten/src/ATen/RegisterQuantizedCPU.cpp, ../aten/src/ATen/RegisterSchema.cpp, ../aten/src/ATen/RegistrationDeclarations.h, ../aten/src/ATen/core/ATenOpList.cpp, ../aten/src/ATen/core/TensorBody.h, ../aten/src/ATen/core/TensorMethods.cpp
[  1%] Building CXX object third_party/fmt/CMakeFiles/fmt.dir/src/os.cc.o
[  1%] Linking CXX static library ../../lib/libfmt.a
[  1%] Built target fmt
[  1%] Building CXX object c10/CMakeFiles/c10.dir/core/CopyBytes.cpp.o
[  1%] Building CXX object third_party/kineto/libkineto/CMakeFiles/kineto_api.dir/src/libkineto_api.cpp.o
[  1%] Building CXX object c10/CMakeFiles/c10.dir/core/DefaultDtype.cpp.o
[  1%] Building CXX object c10/CMakeFiles/c10.dir/core/Device.cpp.o
Scanning dependencies of target aot_compiler
[  1%] Building CXX object caffe2/torch/CMakeFiles/aot_compiler.dir/csrc/jit/mobile/nnc/aot_compiler.cpp.o
[  1%] Built target kineto_api
[  1%] Building CXX object c10/CMakeFiles/c10.dir/core/DeviceType.cpp.o
In file included from /home/sam/tmp/pytorch-v1.10.0/torch/csrc/jit/mobile/nnc/aot_compiler.cpp:1:
In file included from /home/sam/tmp/pytorch-v1.10.0/torch/csrc/jit/mobile/nnc/aot_compiler.h:4:
In file included from /home/sam/tmp/pytorch-v1.10.0/torch/csrc/jit/ir/ir.h:3:
In file included from /home/sam/tmp/pytorch-v1.10.0/torch/csrc/jit/ir/attributes.h:2:
In file included from /home/sam/tmp/pytorch-v1.10.0/aten/src/ATen/ATen.h:9:
In file included from /home/sam/tmp/pytorch-v1.10.0/aten/src/ATen/Context.h:4:
e[1m/home/sam/tmp/pytorch-v1.10.0/aten/src/ATen/Tensor.h:3:10: e[0me[0;1;31mfatal error: e[0me[1m'ATen/core/TensorBody.h' file not founde[0m
#include <ATen/core/TensorBody.h>
e[0;1;32m         ^~~~~~~~~~~~~~~~~~~~~~~~
e[0m[  2%] Building CXX object c10/CMakeFiles/c10.dir/core/DispatchKey.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/core/DispatchKeySet.cpp.o
1 error generated.
make[2]: *** [caffe2/torch/CMakeFiles/aot_compiler.dir/build.make:63: caffe2/torch/CMakeFiles/aot_compiler.dir/csrc/jit/mobile/nnc/aot_compiler.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:1460: caffe2/torch/CMakeFiles/aot_compiler.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[  2%] Building CXX object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/double.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/core/GeneratorImpl.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/core/GradMode.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/core/InferenceMode.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/core/Scalar.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/core/Storage.cpp.o
[  2%] Building CXX object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/complex_single.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/core/StorageImpl.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/core/Stream.cpp.o
[  2%] Building CXX object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/complex_double.cpp.o
[  2%] Built target ATEN_CPU_FILES_GEN_TARGET
[  2%] Building CXX object c10/CMakeFiles/c10.dir/core/TensorImpl.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/core/TensorOptions.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/core/UndefinedTensorImpl.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/core/impl/DeviceGuardImplInterface.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/core/impl/LocalDispatchKeySet.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/core/impl/SizesAndStrides.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/core/thread_pool.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/mobile/CPUCachingAllocator.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/mobile/CPUProfilingAllocator.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/util/Array.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/util/Backtrace.cpp.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/util/C++17.cpp.o
[  2%] Building CXX object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/xerbla.cpp.o
[  2%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/srotm.c.o
[  2%] Building CXX object c10/CMakeFiles/c10.dir/util/DeadlockDetection.cpp.o
[  2%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/srotmg.c.o
[  3%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/drotm.c.o
[  3%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/drotmg.c.o
[  3%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/lsame.c.o
[  3%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/dspmv.c.o
[  3%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/ssbmv.c.o
[  3%] Building CXX object c10/CMakeFiles/c10.dir/util/Exception.cpp.o
[  3%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/chbmv.c.o
[  3%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/sspmv.c.o
[  3%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/zhbmv.c.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/Half.cpp.o
[  4%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/chpmv.c.o
[  4%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/dsbmv.c.o
[  4%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/zhpmv.c.o
[  4%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/dtbmv.c.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/LeftRight.cpp.o
[  4%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/stbmv.c.o
[  4%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/ctbmv.c.o
[  4%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/ztbmv.c.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/Logging.cpp.o
[  4%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/d_cnjg.c.o
[  4%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/r_cnjg.c.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/MathConstants.cpp.o
[  4%] Building C object CMakeFiles/eigen_blas.dir/third_party/eigen/blas/f2c/complexdots.c.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/Metaprogramming.cpp.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/Optional.cpp.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/SmallVector.cpp.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/StringUtil.cpp.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/ThreadLocalDebugInfo.cpp.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/Type.cpp.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/TypeTraits.cpp.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/TypeList.cpp.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/Unicode.cpp.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/UniqueVoidPtr.cpp.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/complex_math.cpp.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/flags_use_gflags.cpp.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/flags_use_no_gflags.cpp.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/intrusive_ptr.cpp.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/numa.cpp.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/signal_handler.cpp.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/thread_name.cpp.o
[  4%] Building CXX object c10/CMakeFiles/c10.dir/util/typeid.cpp.o
[  4%] Linking CXX static library lib/libeigen_blas.a
[  4%] Built target eigen_blas
[  4%] Linking CXX static library ../lib/libc10.a
[  4%] Built target c10
make: *** [Makefile:141: all] Error 2

Context: @cccclai do you have some idea around why this may be failing to build?


Question:
I try to run a seq2seq nmt model follow the pytorch demo application project.
I transform my own model to lite format using this scripts:

encoder_input=torch.tensor([[[429]]])
encoder_length=torch.tensor([1])
decoder_input = torch.tensor([[[123]]])
encoder_output = torch.zeros([1,1,512])
decoder_step = torch.tensor([0])
decoder_output = torch.zeros([1,1,256])
traced_encoder = torch.jit.trace(quant_encoder, (encoder_input, encoder_length))
args = [decoder_input, encoder_output, decoder_step, encoder_length]
kwargs = {“memory_lengths”: encoder_length}
quant_decoder.init_state(encoder_input, None, None)
traced_decoder = torch.jit.trace(quant_decoder, args, strict=False)
traced_generator = torch.jit.trace(quant_generator, (decoder_output))
traced_encoder_optimized = optimize_for_mobile(traced_encoder)
traced_encoder_optimized._save_for_lite_interpreter(“optimized_encoder_150k.ptl”)
traced_decoder_optimized = optimize_for_mobile(traced_decoder)
traced_decoder_optimized._save_for_lite_interpreter(“optimized_decoder_150k.ptl”)
traced_generator_optimized = optimize_for_mobile(traced_generator)
traced_generator_optimized._save_for_lite_interpreter(“optimized_generator_150k.ptl”)

It looks like it completed successfully, at least without reporting an error. Then I try to run my app throw android like this:

    int input_length = inputs.length;
    final long[] inputShape = new long[]{1, input_length, 1};
    final long[] inputlengthShape = new long[]{input_length};;

    final long[] outputsShape = new long[]{MAX_LENGTH, HIDDEN_SIZE};
    final FloatBuffer outputsTensorBuffer =
            Tensor.allocateFloatBuffer(MAX_LENGTH  * HIDDEN_SIZE);
    LongBuffer inputlengthTensorBuffer = Tensor.allocateLongBuffer(1);
    Tensor inputlengthTensor = Tensor.fromBlob(inputlengthTensorBuffer, inputlengthShape);

    for (int i=0; i<inputs.length; i++) {
        LongBuffer inputTensorBuffer = Tensor.allocateLongBuffer(input_length);
        inputTensorBuffer.put(inputs[i]);

        Tensor inputTensor = Tensor.fromBlob(inputTensorBuffer, inputShape);
        Log.i("INFO input tensor", Arrays.toString(inputTensor.getDataAsLongArray()));
        Log.i("INFO input tensor shape", Arrays.toString(inputTensor.shape()));
        Log.i("INFO input length tensor shape", Arrays.toString(inputlengthTensor.shape()));
        IValue temp = IValue.from(inputTensor);
        Log.i("INFO input tensor test", Arrays.toString(temp.toTensor().getDataAsLongArray()));
        final IValue[] outputTuple = mModuleEncoder.forward(IValue.from(inputTensor), IValue.from(inputlengthTensor)).toTuple();

The input is only one integer representing a character. But it always return an error:

com.facebook.jni.CppException: Expected batch2_sizes[0] == bs && batch2_sizes[1] == contraction_size to be true, but got false.  (Could this error message be improved?  If so, please report an enhancement request to PyTorch.)

It’s a litte hard to debug this as I can’t get the middle result. I’ve been struggling with this issue for two days…
Any response is appreciate !

Context:
Do you see any log outputs and could use it to narrow down the lines of code raising the error?
If so, log the shapes of the used tensors and post the operation here, so that we could have a look why the shape mismatch is raised.


Question:
Hello
I am developing with reference to the pytorch object detection example.
I am trying to detect an object using pytorch model in android.
It receives an image in real time, receives this image as a bitmap, and displays a specific object as the result.
However, the TensorImageUtils.bitmapToFloat32Tensor function is time consuming and very slow compared to live images.
bitmap = Bitmap.createBitmap(bmp, 0, 0, bmp.width, bmp.height, matrix, true)
        var resizedBitmap = Bitmap.createScaledBitmap(bitmap,PrePostProcessor.mInputWidth,PrePostProcessor.mInputHeight,true)
        inputTensor = TensorImageUtils.bitmapToFloat32Tensor(resizedBitmap,PrePostProcessor.NO_MEAN_RGB,PrePostProcessor.NO_STD_RGB)
        outputTuple  = mModule!!.forward(IValue.from(inputTensor)).toTuple()
        outputTensor = (outputTuple as Array<out IValue>?)?.get(0)!!.toTensor()
        output = outputTensor!!.dataAsFloatArray

Is there any way to solve this? Should I force it to use the gpu? If so, how?

Context:
How large is your image? Do you have any latency number?
bitmapToFloat32Tensor is a function from torchvision. I don’t think it can run on GPU


Question:
Hello,
I’m tring to calculate the cosine similarity between two tensors of different shapes:
tensor1: torch.Size([15, 24, 4, 120])
tensor2: torch.Size([5608, 4, 120])
To make them have the same dimension, I used unsqueeze to make them both of size
torch.Size([15, 24, 5608, 4, 120])
However, this new tensor makes me run out of memory because it has 969M elements and takes 3.8Gb.
Is there a better way to calculate cosine similarity without creating this giant tensor?

Context:
@Yunchao_Liu
you can try something like
tensor1 = torch.rand([15, 24, 4, 120])
tensor2 = torch.rand([5608, 4, 120])
cos = nn.CosineSimilarity(dim=0)

def custom_cos(cur_tensor):
    return cos(cur_tensor, tensor2)

results = map(custom_cos, tensor1.view(-1, 4, 120))
results = torch.concat(list(results)).view(15,24,4,120)
results.size()

If you have the latest version you can also use
https://pytorch.org/tutorials/prototype/vmap_recipe.html 2


Performance issue of RTX 3070 compared to 2070 SUPER

Question:
Hello,
I have a flask server running inference for several different models. Two identical computers that only differs in terms of GPU, one RTX3070 and the other 2070 SUPER. Both computers have i9-10900X and nvme2 SSDs and same RAM. Both computers are up to date with every library and pytorch 1.10.0 with 11.3 cuda and 8.2.0 cudnn. So in summary, they are exactly the same with everything else.
However, RTX 3070 performs significantly slower and uses much more VRAM (~7.6GB) than 2070SUPER (~6GB VRAM). 2070SUPER can handle %50 more load and still perform better.
Are there any known issues regarding the performance issues on 3070?
Note: I use Transformers and Attention networks which I heard should run faster on 30** series GPUs.
Note2: I use half precision for CNN models as well. Can the issue be related with fp16?
Note3: I regularly clear gpu cache release some memory, otherwise VRAM keeps bloating under heavy load.
Note4: Flask runs multithreaded if that helps.
Note5: This issue is impossible to reproduce due to complexity of the code.

Context:
I’m not sure what “Note 3” means and how you are releasing the memory, but in case you are clearing the cache this would hit the performance.
Note 5 makes it quite hard to give you a valid answer. Would it be possible to get a model with the input shapes showing the performance difference?


Question:
I have a tiny CNN model that gives this summary:
----------------------------------------------------------------
        Layer (type)               Output Shape         Param #
================================================================
            Conv2d-1         [-1, 32, 222, 222]             896
              ReLU-2         [-1, 32, 222, 222]               0
         MaxPool2d-3         [-1, 32, 111, 111]               0
            Conv2d-4         [-1, 32, 109, 109]           9,248
              ReLU-5         [-1, 32, 109, 109]               0
         MaxPool2d-6           [-1, 32, 54, 54]               0
            Conv2d-7           [-1, 32, 52, 52]           9,248
              ReLU-8           [-1, 32, 52, 52]               0
         MaxPool2d-9           [-1, 32, 26, 26]               0
          Flatten-10                [-1, 21632]               0
           Linear-11                    [-1, 1]          21,633
          Sigmoid-12                    [-1, 1]               0
================================================================
Total params: 41,025
Trainable params: 41,025
Non-trainable params: 0
----------------------------------------------------------------
Input size (MB): 0.57
Forward/backward pass size (MB): 35.24
Params size (MB): 0.16
Estimated Total Size (MB): 35.97
----------------------------------------------------------------

The input images are 3x224x224 and the batch size is 16. When I start training the model (with torch.optim.SGD), I get this:
RuntimeError: CUDA out of memory. Tried to allocate 11.89 GiB (GPU 0; 8.00 GiB total capacity; 1.14 GiB already allocated; 4.83 GiB free; 1.14 GiB reserved in total by PyTorch)

Why does it try to allocate so much memory for such a small network? Thanks.

Context: I found the bug which was trivial: I was loading all the validation images into a single batch which was too big. I would delete this post but don’t see any option to do that.


Dataloader num_workers relate to gpu memory?

Question: If I increase num_workers from 0 to 4, then could be cuda out of memory problem solved? RAM is enough but memory is exploded…help me please

Context:
No, increasing num_workers in the DataLoader would use multiprocessing to load the data from the Dataset and would not avoid an out of memory on the GPU.
To solve the latter you would have to reduce the memory usage by e.g. reducing the batch size or by using e.g. torch.utils.checkpoint.


Why CUDACachingAllocator limited block shareing inside stream?

Question:
bool get_free_block(AllocParams& p) {
    BlockPool& pool = *p.pool;
    auto it = pool.lower_bound(&p.search_key);
    if (it == pool.end() || (*it)->stream != p.stream())
      return false;
    p.block = *it;
    pool.erase(it);
    return true;
  }

If block is allocated by one stream, and splited to one free block, why can’t we use it in aother stream?
When CudaMalloc is host blocking.

Context: Anyone has idea on it?


Creating tensors on CPU and measuring the memory consumption?

Question:
Let’s say that I have a PyTorch tensor that I’m loading onto CPU. I would now like to experiment with different shapes and how they affect the memory consumption, and I thought the best way to do this is creating a simple random tensor and then measuring the memory consumptions of different shapes. However, while attempting this, I noticed anomalies and I decided to simplify the task further.
I’m creating a 3 GB PyTorch tensor and want to measure its memory consumption with the psutil module. In order to get some statistics, I do this ten times in a for loop and consider the mean and std. I also move the tensor to GPU and then use PyTorch functions to measure the allocated and totally reserved memory on GPU.
My code is this:
import torch
import psutil

if __name__ == '__main__':
    resident_memories = []
    for i in range(10):
        x = torch.ones((3, 1024, 1024, 1024), dtype=torch.uint8)
        resident_memory = psutil.Process().memory_info().rss/1024**2
        resident_memories.append(resident_memory)
        del x, resident_memory
    print('Average resident memory [MB]: {} +/- {}'.format(torch.mean(torch.tensor(resident_memories)), torch.std(torch.tensor(resident_memories))))
    del resident_memories

    device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')
    alloc_memories = []
    reserved_memories = []
    for i in range(10):
        x = torch.ones((3, 1024, 1024, 1024), dtype=torch.uint8).to(device)
        alloc_memory = torch.cuda.memory_allocated(device=device)/1024**2
        reserved_memory = torch.cuda.memory_reserved(device=device)/1024**2
        alloc_memories.append(alloc_memory)
        reserved_memories.append(reserved_memory)
        del x, alloc_memory
    print('By tensors occupied memory on GPU [MB]: {} +/- {}\nCurrent GPU memory managed by caching allocator [MB]: {} +/- {}'.format(
          torch.mean(torch.tensor(alloc_memories)), torch.std(torch.tensor(alloc_memories)), torch.mean(torch.tensor(reserved_memories)),
          torch.std(torch.tensor(reserved_memories)))
    )

I obtain the following output:
Average resident memory [MB]: 4028.602783203125 +/- 0.06685283780097961
By tensors occupied memory on GPU [MB]: 3072.0 +/- 0.0
Current GPU memory managed by caching allocator [MB]: 3072.0 +/- 0.0

I’m executing this code on a cluster, but I also ran the first part on the cloud and I mostly observed the same behavior. When I ran this on the cluster, it was the only job on the CPU, so other jobs should (hopefully) not affect the memory consumption.
I’d have two quick questions:
i) Why is  psutil.Process().memory_info().rss inaccurate when measuring the memory of a 3 GB tensor?
ii) How can we (correctly) measure the memory of tensors on CPU? One use case might be that the tensor is so huge that moving it onto a GPU might cause a memory error.

Context: @ptrblck Do you think that you could help, please?


Question:
I don’t know what this means.
If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

Context:
Take a look at the Memory Management docs 52 which explain how the caching memory allocator works.
The last section explains how the env variable can be used to prevent fragmentation in case your workload is suffering from it.


Question: I was hoping there is a quick method to remove all pytorch from the GPU?

Context: You could delete all tensors, parameters, models etc. and call empty_cache() afterwards to remove all allocations created by PyTorch. To also remove the CUDA context, you would have to shut down the Python session.


No process is running (nivdia-smi ) but Runntime Error memory is full

Question:
Below is the result of nivdia-smi command. Which shows that no process is running. but when i try to run my code it says
RuntimeError: CUDA out of memory. Tried to allocate 1.02 GiB (GPU 3; 7.80 GiB total capacity; 6.24 GiB already allocated; 258.31 MiB free; 6.25 GiB reserved in total by PyTorch)
nivdia-smi  results:
Thu Dec 23 11:52:33 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 455.23.05    Driver Version: 455.23.05    CUDA Version: 11.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GeForce RTX 208...  Off  | 00000000:05:00.0 Off |                  N/A |
| 30%   43C    P8    19W / 250W |      3MiB /  7981MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   1  GeForce RTX 208...  Off  | 00000000:06:00.0 Off |                  N/A |
| 27%   41C    P8     3W / 250W |      3MiB /  7982MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   2  GeForce RTX 208...  Off  | 00000000:09:00.0 Off |                  N/A |
| 27%   39C    P8     2W / 250W |      3MiB /  7982MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   3  GeForce RTX 208...  Off  | 00000000:0A:00.0 Off |                  N/A |
| 27%   35C    P8     3W / 250W |      3MiB /  7982MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

Note:
1- I already tried: torch.cuda.empty_cache()
2- This is a shared sever and I have user access not root access.

Context:
The error message explains that you are running out of memory and cannot allocate the desired ~1GB.
Your total GPU memory is given as 7.8GB, 6.25GB are already reserved by PyTorch, ~260MB are free, the rest is used by the CUDA context (since you’ve made sure no other applications are running and using the device).


Question:
When I run my code, I get this runtime error below:
CUDA out of memory. Tried to allocate 88.00 MiB (GPU 0; 8.00 GiB total capacity; 6.04 GiB already allocated; 0 bytes free; 6.17 GiB reserved in total by PyTorch)
I don’t understand why it says 0 bytes free;
Maybe I should have at least 6.17 - 6.04 = 0.13 GiB free?
I’m afraid if it’s not really a memory shortage problem.

Context: You are trying to allocate 88MB. ~130MB are in the cache, but are not a contiguous block, so cannot be used to store the needed 88MB. 0B are free, which shows that the rest of your device memory is used to store the CUDA context, memory for other applications etc.


Question:
Hi
I have a big issue with memory. I am developing a big application with GUI for testing and optimizing neural networks. The main program is showing the GUI, but training is done in thread. In my app I need to train many models with different parameters one after one. To do this I need to create a model for each attempt. When I train one I want to delete it and train new one, but I cannot delete old model. I am trying to do something like this:
del model
torch.cuda.empty_cache()

but GPU memory doesn’t change,
then i tried to do this:
model.cpu()
del model

When I move model to CPU, GPU memory is freed but CPU memory increase.
In each attempt of training, memory is increasing all the time. Only when I close my app and run it again the all memory is freed.
Is there a way to delete model permanently from GPU or CPU?

Context:
I cannot reproduce the issue using a recent master build:
model = models.resnet18()
print(torch.cuda.memory_allocated())
> 0

model.cuda()
print(torch.cuda.memory_allocated())
> 46861312

del model
print(torch.cuda.memory_allocated())
> 0


Why reserved_memory+allocated memory is bigger than the memory shown in nvidia-smi?

Question:
It seems like that reserved_memory+allocated memory is bigger than the memory shown in nvidia-smi. My allocated mem is 3199MB and my reserved mem is 8766 MB, which equals 11965MB, but the mem shown in my nvidia-smi is 11131MB.
By the way, there is another question, I met an OOM when trying to allocate about 1000MB space but there was still about 8766MB in reserved space. The reserved space is designed to be reused, why is there still an OOM error?

Context: The reserved memory reports the allocated memory and the cached memory so you shouldn’t sum them together. Take a look at the memory management docs 5 for more information.


Question:
Hi, I have a memory leakage problem with the code below:
import torch
import torch.nn as nn

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.c, self.k, self.p, self.x = 256, 3, 1, 2304
        self.n, self.h, self.w = 0, 0, 0
        self.unfold = nn.Unfold(self.k, padding=self.p)
        self.conv = nn.Conv2d(self.c * self.k * self.k, self.x, 1, padding=0, groups=self.c * self.k * self.k)

    def forward(self, x: torch.Tensor):
        self.n, self.h, self.w = x.size(0), x.size(2), x.size(3)
        c1 = x.view(self.n, self.c, 1, self.h, self.w)

        c2 = self.unfold(x)
        c2 = c2.view(self.n, self.c, self.k * self.k, self.h, self.w)

        out = c1 + c2
        out = out.view(self.n, self.c * self.k * self.k, self.h, self.w)
        return self.conv(out)

if __name__ == '__main__':
    try:
        net = Net().cuda()
        x = torch.randn((1024, 256, 32, 32)).cuda()
        out = net(x)
        print(out)
    except Exception as ex:
        print(f' > CUDA memory: {torch.cuda.memory_allocated() / 1024 ** 3} GiB')
        del x, net
        print(ex)
    print(f' > CUDA memory: {torch.cuda.memory_allocated() / 1024 ** 3} GiB')

Here I define a network with depth-wise convolution as the last layer. I’m using 24GB RTX 3090, so when it runs into self.conv(out), it will surely be out of CUDA memory due to the large tensors. Then we can catch the exception and delete the references of x and net, the expected CUDA memory after del x, net should be zero but I got 9 GB instead, which means that there’s a memory leakage with a failed-to-allocate-output depth-wise convolution. If I change the group number of self.conv into something else, the CUDA memory will be correctly zero.

Context:
I get the expected result:
 > CUDA memory: 19.000017166137695 GiB
CUDA out of memory. Tried to allocate 9.00 GiB (GPU 0; 23.69 GiB total capacity; 19.00 GiB already allocated; 1.48 GiB free; 19.00 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
 > CUDA memory: 0.0 GiB

but think that these “hacks” might easily fail (as seen in your setup) and would thus recommend to set a proper batch size before starting the training.
In any case, you could check if deleting out (if it’s even created) might solve your issue.


Question:
>>> out = torch.randn(100,100)
>>> out.cuda()
>>> torch.cuda.memory_allocated(0)
40448
>>> out1 = torch.randn(100,100)
>>> torch.cuda.memory_allocated(0)
0

Why is this happening? I expected the output of totorch.cuda.memory_allocated(0) be 40448 still, because out1 is totally different to out .
Note that,
>>> id(out)
139989884735632
>>> id(out1)
139989884693456

Note2
>>> out = torch.randn(100,100)
>>> out.cuda()
>>> out1 = torch.randn(90,90)
>>> torch.cuda.memory_allocated(0)
40448

Context:
The underscore (_) is a special variable which will get the output of the last operation as seen here:
a = 5

a
Out[2]: 5

6
Out[3]: 6

_
Out[4]: 6

_ + _
Out[5]: 12

If you are stepping through the code in a REPL env I would guess that in particular this variable can hold a reference to the CUDATensor and could thus report the GPU usage. On the other hand, if you execute the entire script, the memory usage is shown as 0. which is expected since you didn’t assign the CUDATensor to any variable so it will be discarded.


How to convert a pytorch nn module to float 64

Question: I want a simple technique that will convert a pytorch nn.module to a float 64 model.

Context: To transform all parameters and buffers of a module to float64 tensors, use model.double().


Why aren’t inputs to conv1d channels last?

Question:
Just to replicate a historic approach (Kim, 2014), I am experimenting with conv1d for a sentence classification task. Embedding size for each word = 300. Sequence length (i.e. number of words in a sentence) =  512. So my input sentence is [512, 300]. (Let us omit batch size for this discussion). This way I can refer to each word in my input sentence as input[word_index].  A conv layer needs to convolve over sequence length (say, to get bi-grams or tri-grams). This means your embedding needs to be seen as ‘in_channels’.
Now, after digging around SO answers, it turns out, conv1d expects my input to have channels first, i.e. a sentence needs to be [300, 512].
This breaks the row-major convention which are historic defaults by C, NumPy, etc.
For output, as a diagram shows in Kim’s paper, one might expect sequence length to be the first index and channels (corresponding to your multiple filters) to be the second one. nn.conv1d gives output in [out_channels, sequence_length_with_padding_stride_calculation] format.
Why does PyTorch favor channels first?
If it’s a matter of convention, shouldn’t there be an easy way to choose your convention?
If it’s because CuDNN prefers NCHW, can PyTorch not internally change input?
The to and contiguous approach outlined here is convoluted.  Why would you want devs to explicitly convert their input and their models? Can we not have a parameter in model declaration that decides channels_first or channels_last and the input is processed accordingly? If you want to maintain backwards compatibility, the parameter might have channels_first as default.
P.S. the conv1d documentation could greatly benefit if there is an explanation of this below the example, and possibly, if your input is channels_last, how to convert it and have contiguous memory allocation. If this needs to be added only in conv1d, I can raise a PR. If this needs explanation across multiple pages, a longer discussion might be helpful.

Context:
Yash_Jakhotiya:

Why does PyTorch favor channels first?

For historic performance reasons, if I’m not mistaken.

 Yash_Jakhotiya:

If it’s a matter of convention, shouldn’t there be an easy way to choose your convention?
If it’s because CuDNN prefers NCHW, can PyTorch not internally change input?

PyTorch can change the memory layout via to(memory_format=torch.channels_last), which is the preferred format for mixed-precision convolutions using TensorCores.

 Yash_Jakhotiya:

Why would you want devs to explicitly convert their input and their models? Can we not have a parameter in model declaration that decides channels_first or channels_last and the input is processed accordingly?

Wouldn’t this already be the case or are you concerned about having to add the to() operation to the input as well? Under the hood, the parameter layout might be respected and the inputs thus transformed for you, but explicit transformations are preferred over implicit ones.


Question:
I am training a stacked GRU with a linear output layer. I have verified the input size is 3 dim. The code is giving me the following error: TypeError: relu(): argument ‘input’ (position 1) must be Tensor, not tuple
I have written code to convert my input numpy arrays to torch tensors but I am still getting that error.

class CustomDataset(Dataset):
def init(self, X_data, Y_data):
super(CustomDataset, self).init()
self.X_data = X_data.float()
self.Y_data = Y_data.float()

# Initializes the data and preprocessing.

def __getitem__(self, index):
    return self.X_data[:,:,:], self.Y_data[:,:,:]
# Returns data (input and output) in batches.
# There is an error in this part of the code.
def __len__ (self):
    return len(self.X_data[:,1,1])
# Returns the size of the input data.

inputdata = np.load(‘4thin.npy’)
outputdata = np.load(‘4thout.npy’)

trainX = inputdata[:208,:,:10]
trainX = torch.from_numpy(trainX)
print(trainX.shape)

trainY = outputdata[:208,:,:8]
trainY = torch.from_numpy(trainY)
print(trainY.shape)

testX = inputdata[208:,:,:10]
testX = torch.from_numpy(testX)
print(testX.shape)

testY = outputdata[208:,:,:8]
testY = torch.from_numpy(testY)
print(testY.shape)
#select a subset of the features from the data for testing of the script
#Loads in the input and output numpy arrays and splits into training and validation datasets.

train_dataloader = CustomDataset(trainX, trainY)
test_dataloader = CustomDataset(testX, testY)

The size of each of the tensors is the following:

torch.Size([208, 3, 10])
torch.Size([208, 1, 8])
torch.Size([208, 3, 10])
torch.Size([208, 1, 8])

I set up the program to print the x tensor and got the following:

tensor([[[  0.,   0.,   0.,  …,   0.,   0.,   0.],
[  0.,   0.,   0.,  …,   0.,   0.,   0.],
[ 15.,   5.,  34.,  …,   4.,  11.,   1.]],
[[  0.,   0.,   0.,  …,   7.,   6.,   7.],
[  6.,  15.,   0.,  …,   0.,   7.,   0.],
[  1.,   5.,  17.,  …,  61.,   9.,   0.]],
[[ 58.,  58.,  35.,  …,  87.,  54.,  38.],
[  0.,   0.,   0.,  …,   0.,   0.,   0.],
[  8.,   0.,   0.,  …,   0.,   0.,   0.]],
…,
[[  2.,   0.,   8.,  …,   3.,   2.,   2.],
[  4.,   4.,  30.,  …,   5.,   0.,  23.],
[125., 145.,  97.,  …, 179., 704., 140.]],
[[  0.,   0.,   0.,  …,   1.,   0.,   0.],
[ 78.,   0.,  43.,  …,  62.,   0.,  64.],
[ 45.,  88.,   0.,  …,   0.,  15.,   0.]],
[[  0.,   0.,   0.,  …,   0.,   0.,   0.],
[  0.,   0.,   0.,  …,   0.,   0.,   0.],
[112.,  84., 104.,  …, 133., 125.,  75.]]])

The script seems to think this is a tuple and not a tensor. Help would be much appreciated!

Context:
Could you post the model definition, please?
Sometimes these errors are caused by an accidental training comma in a line of code, e.g.
x = self.layer(x),
x = F.relu(x)

(note the comma in the first line of code).


Out of Memory and Can’t Release GPU Memory

Question:
I use try-catch to enclose the forward and backward functions, and I also delete all the tensors after every batch.
            try:
                decoder_output, loss = model(batch)
                if Q.qsize() > 0:
                    break
                optimizer.zero_grad()
                loss.backward()
                optimizer.step()
                train_loss.append(loss.mean().item())
                del decoder_output, loss
            except Exception as e:
                optimizer.zero_grad()
                for p in model.parameters():
                    if p.grad is not None:
                        del p.grad
                torch.cuda.empty_cache()
                oom += 1
            del batch
            for p in model.parameters():
                if p.grad is not None:
                    del p.grad
            torch.cuda.empty_cache()

The training process is normal at the first thousands of steps, even if it got OOM exception, the exception will be catched and the GPU memory will be released.
But after I trained thousands of batches, it suddenly keeps getting OOM for every batch and the memory seems never be released anymore.
It’s so weird to me, is there any suggestions? (I’m using distributed data parallel)

Context:
Here is the code before the try-catch, i.e., the code to prepare input data with train_loader:
        for i, batch in enumerate(train_loader):
            batch = {k: v.squeeze(0).cuda(non_blocking=True) for (k, v) in batch.items()}
            batch['rem_father_index'] = torch.split(batch['rem_father_index'], batch['rem_root_num'].tolist(), dim=0)
            batch['rem_father_index'] = [l.tolist() for l in batch['rem_father_index']]
            batch['tree_sizes'] = batch['tree_sizes'].tolist()


Understanding error msg “view size is not compatible with input tensor’s size and stride”

Question:
Ideally, I would like to execute something like
t = torch.zeros([4, 3, 64, 64])
t[:, :, ::8, ::8].view(4, -1)
but that produces the error
RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.

Unfortunately, I can’t use .reshape() or .contiguous() because of memory consumption. This code is called too often to make a copy of the tensor each time. Instead I would like to create one big tensor and slice it each time.
Is there some way to use .transpose() or something similar in combination with the above .view() to achieve my goal? Is there a way to get a more detailed error message to understand which dimension exactly is the problem?Thanks a lot in advance!

Context:
Yes, sorry, output from stride() should be reasoned about together with tensor size. Here, your last dimension says: increase pointer by 8 (last stride) 8 times (last dim size), but the third dimension wants to to take steps of 512 elements (it is skipping 7 segments altogether).
Stride of 8 corresponds to t[:,:,:,::8], where you have stride[i] = stride[i+1] * size[i+1]


Uncollected object references from torch.load() cause memory growth

Question:
Hi all,
I’m currently working on training a model on a rather large dataset ( ~1 billion examples in total). To store and load this dataset, I’m employing the Webdataset library (https://github.com/webdataset/webdataset). I am storing preprocessed examples as Pytorch tensors in a tar file, per the Webdataset spec. I am running into a memory issue after a long period of training where the machine performing the reading/loading operation runs out of memory. After a bit of debugging, it seems to be caused by some unexpected (at least to me) behavior with the Pytorch serialization code.
Since each tensor is stored as a separate file, then tarred together, each must be loaded individually, meaning that ~1 billion torch.load() operations take place per epoch. I traced memory usage using the tracemalloc utility and discovered that some part of this serialization process is creating objects that are not garbage collected even after the tensor read from disk goes out of scope (or in the example below, is manually deleted).
I have included a minimal example below to show these objects. The example creates a dummy tensor, then reads it into memory repeatedly. The memory usage is queried in 10,000 step intervals. Given that that the tensor goes out of scope immediately after it is read with torch.load(), I would not expect to see any objects from torch/serialization.py to still be in scope. However, I instead see ~10,000 new objects created each time the memory is queried (every 10,000 steps, see example output below). These objects also seem to never be released and cause constant linear memory growth.
Is there something I am missing here? Is there a way to remove references to these objects so they can be collected? Even though they are only 28B each, on average, they still seem to be causing memory use to grow linearly until the machine is out of memory (usually after many millions of examples are read from disk on a high-mem training machine).
Thank you in advance for any and all help you can offer!
Best,
Tyler

System info

Ubuntu 18.04.1
Python 3.8
Pytorch 1.9.0

Minimal reproducible example:
import torch
import tracemalloc

if __name__ == '__main__':
    # Create a dummy Tensor serialized to disk
    rand = torch.rand((20, 10))
    torch.save(rand, 'test.pth')

    # Start tracemalloc
    tracemalloc.start(30)
    old_snapshot = tracemalloc.take_snapshot()

    for i in range(100000):
        # Load the Tensor,
        test_tensor = torch.load('test.pth')

        # Immediately delete reference to test_tensor
        del test_tensor

        if i % 10000 == 0 and i != 0:
            # Take snapshot
            snapshot = tracemalloc.take_snapshot()

            # Print changes in memory consumption
            print(f'################# STEP {i} #################')
            for stat in snapshot.compare_to(old_snapshot, 'lineno')[:2]:
                print(str(stat))
            print('############################################')

            # Save snapshot
            old_snapshot = snapshot

Example output:
################# STEP 10000 #################
.../python3.8/site-packages/torch/serialization.py:845: size=274 KiB (+274 KiB), count=10005 (+10005), average=28 B
.../python3.8/site-packages/torch/serialization.py:242: size=274 KiB (+274 KiB), count=10003 (+10003), average=28 B
################# STEP 20000 #################
.../python3.8/site-packages/torch/serialization.py:845: size=547 KiB (+273 KiB), count=20005 (+10000), average=28 B
.../python3.8/site-packages/torch/serialization.py:242: size=547 KiB (+273 KiB), count=20003 (+10000), average=28 B
################# STEP 30000 #################
.../python3.8/site-packages/torch/serialization.py:242: size=820 KiB (+273 KiB), count=30003 (+10000), average=28 B
.../python3.8/site-packages/torch/serialization.py:845: size=821 KiB (+273 KiB), count=30004 (+9999), average=28 B

The two lines being referenced from torch/serialization.py above are:

      github.com

    pytorch/pytorch/blob/v1.9.0/torch/serialization.py#L845

def _load(zip_file, map_location, pickle_module, pickle_file='data.pkl', **pickle_load_args):
              restore_location = _get_restore_location(map_location)

    loaded_storages = {}

    def load_tensor(data_type, size, key, location):
                  name = f'data/{key}'
                  dtype = data_type(0).dtype

        storage = zip_file.get_storage_from_record(name, size, dtype).storage()
                  loaded_storages[key] = restore_location(storage, location)

    def persistent_load(saved_id):
                  assert isinstance(saved_id, tuple)
                  typename = _maybe_decode_ascii(saved_id[0])
                  data = saved_id[1:]

        assert typename == 'storage', \
                      f"Unknown typename for persistent_load, expected 'storage' but got '{typename}'"
                  data_type, key, location, size = data

and

      github.com

    pytorch/pytorch/blob/v1.9.0/torch/serialization.py#L242

                  if 'w' in mode:
                      return _open_buffer_writer(name_or_buffer)
                  elif 'r' in mode:
                      return _open_buffer_reader(name_or_buffer)
                  else:
                      raise RuntimeError(f"Expected 'r' or 'w' in mode but got {mode}")

class _open_zipfile_reader(_opener):
              def __init__(self, name_or_buffer) -> None:
                  super(_open_zipfile_reader, self).__init__(torch._C.PyTorchFileReader(name_or_buffer))

class _open_zipfile_writer_file(_opener):
              def __init__(self, name) -> None:
                  super(_open_zipfile_writer_file, self).__init__(torch._C.PyTorchFileWriter(str(name)))

    def __exit__(self, *args) -> None:
                  self.file_like.write_end_of_file()

Context: del x doesn’t guarantee garbage collection in Python, you need to try with gc.collect()


Question:
I am new in Pytorch. I got this error when I was trying to train the GCN model on my custom dataset.
The error:
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
/tmp/ipykernel_61/1753868253.py in <module>
     30 # Run for 200 epochs (range is exclusive in the upper bound)
     31 for epoch in range(1, 201):
---> 32     train()
     33     train_acc = test(train_loader)
     34     test_acc = test(test_loader)

/tmp/ipykernel_61/1753868253.py in train()
     10     for data in train_loader:
     11       print('x_shape:',data.x.shape)
---> 12       out = model(data.x.float(), data.edge_index, data.batch)
     13       loss = criterion(out, data.num_classes)
     14       loss.backward()

RuntimeError: [enforce fail at CPUAllocator.cpp:68] . DefaultCPUAllocator: can't allocate memory: you tried to allocate 8609587200 bytes. Error code 12 (Cannot allocate memory)

My code:
train_loader = DataLoader(trainset, batch_size=64, shuffle=True)
test_loader = DataLoader(testset, batch_size=64, shuffle=False)

# Define our GCN class as a pytorch Module
class GCN(torch.nn.Module):
    def __init__(self, hidden_channels):
        super(GCN, self).__init__()
        # We inherit from pytorch geometric's GCN class, and we initialize three layers
        self.conv1 = GCNConv(data.num_features, hidden_channels)
        self.conv2 = GCNConv(hidden_channels, hidden_channels)
        self.conv3 = GCNConv(hidden_channels, hidden_channels)
        # Our final linear layer will define our output
        self.lin = Linear(hidden_channels, 8)

    def forward(self, x, edge_index, batch):
      # 1. Obtain node embeddings
      x = self.conv1(x, edge_index)
      x = x.relu()
      x = self.conv2(x, edge_index)
      x = x.relu()
      x = self.conv3(x, edge_index)

      # 2. Readout layer
      x = global_mean_pool(x, batch)  # [batch_size, hidden_channels]

      # 3. Apply a final classifier
      x = F.dropout(x, p=0.5, training=self.training)
      x = self.lin(x)
      return x

model = GCN(hidden_channels=64)

Context:
Samah_Abu_saleem:

DefaultCPUAllocator: can't allocate memory: you tried to allocate 8609587200 bytes.

The error message seems clear?
That your model is needing more memory than the GPU memory size?


TypeError: empty(): argument ‘size’ must be tuple of ints, but found element of type Tensor at pos 1

Question:
I try to implement GCN on my custom dataset, but I got error:
my code:
# Define our GCN class as a pytorch Module
class GCN(torch.nn.Module):
    def __init__(self, hidden_channels):
        super(GCN, self).__init__()
        # We inherit from pytorch geometric's GCN class, and we initialize three layers
        self.conv1 = GCNConv(data.num_features, hidden_channels)
        self.conv2 = GCNConv(hidden_channels, hidden_channels)
        self.conv3 = GCNConv(hidden_channels, hidden_channels)
        # Our final linear layer will define our output
        self.lin = Linear(hidden_channels, data.num_classes)

    def forward(self, x, edge_index, batch):
      # 1. Obtain node embeddings
      x = self.conv1(x, edge_index)
      x = x.relu()
      x = self.conv2(x, edge_index)
      x = x.relu()
      x = self.conv3(x, edge_index)

      # 2. Readout layer
      x = global_mean_pool(x, batch)  # [batch_size, hidden_channels]

      # 3. Apply a final classifier
      x = F.dropout(x, p=0.5, training=self.training)
      x = self.lin(x)
      return x

model = GCN(hidden_channels=16)
print(model)

The error:
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/tmp/ipykernel_59/1705364933.py in <module>
     32       return x
     33
---> 34 model = GCN(hidden_channels=16)
     35 print(model)

/tmp/ipykernel_59/1705364933.py in __init__(self, hidden_channels)
     14         self.conv3 = GCNConv(hidden_channels, hidden_channels)
     15         # Our final linear layer will define our output
---> 16         self.lin = Linear(hidden_channels, data.num_classes)
     17
     18     def forward(self, x, edge_index, batch):

/usr/local/lib/python3.8/dist-packages/torch/nn/modules/linear.py in __init__(self, in_features, out_features, bias, device, dtype)
     83         self.in_features = in_features
     84         self.out_features = out_features
---> 85         self.weight = Parameter(torch.empty((out_features, in_features), **factory_kwargs))
     86         if bias:
     87             self.bias = Parameter(torch.empty(out_features, **factory_kwargs))

TypeError: empty(): argument 'size' must be tuple of ints, but found element of type Tensor at pos 1

Context:
nn.Linear should be defined by two integers.
In short
data.num_classes = n instead of data.num_classes = torch.tensor([0, 1, 3, 2])


Question:
I want to implement GCN on my data.
the data looks like this:
Data(pos=[4, 2], x=[4, 1080, 1920, 3], y=[4], edge_index=[2, 12], edge_attr=[12, 1])
my code:
# Set optimizer (adam)
optimizer = torch.optim.Adam(model.parameters(), lr=0.01)
# Define loss function
criterion = torch.nn.CrossEntropyLoss()

# Initialize train function
def train():
    model.train()

    for data in trainset:
      out = model(data.x.view(-1, 3), data.edge_index, data.batch)
      loss = criterion(out, data.y)
      loss.backward()
      optimizer.step()
      optimizer.zero_grad()

# Define test function
def test(loader):
  model.eval()

  correct = 0
  for data in loader:
      out = model(data.x, data.edge_index, data.batch)
      pred = out.argmax(dim=1)
      correct += int((pred == data.y).sum())
  return correct / len(loader.dataset)

# Run for 200 epochs (range is exclusive in the upper bound)
for epoch in range(1, 201):
    train()
    train_acc = test(trainset)
    test_acc = test(testset)
    print(f'Epoch: {epoch:03d}, Train Acc: {train_acc:.4f}, Test Acc: {test_acc:.4f}')

the error:
RuntimeError                              Traceback (most recent call last)
/tmp/ipykernel_135/1724653464.py in <module>
     29 # Run for 200 epochs (range is exclusive in the upper bound)
     30 for epoch in range(1, 201):
---> 31     train()
     32     train_acc = test(trainset)
     33     test_acc = test(testset)

/tmp/ipykernel_135/1724653464.py in train()
      9
     10     for data in trainset:
---> 11       out = model(data.pixel_values.view(-1, 3), data.edge_index, data.batch)
     12       loss = criterion(out, data.detection_score)
     13       loss.backward()

/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
   1100         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1101                 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1102             return forward_call(*input, **kwargs)
   1103         # Do not call functions when jit is used
   1104         full_backward_hooks, non_full_backward_hooks = [], []

/tmp/ipykernel_135/2620939698.py in forward(self, x, edge_index, batch)
     19       # 1. Obtain node embeddings
     20       px = data.pixel_values.view(-1, 3)
---> 21       x = self.conv1(px, edge_index)
     22       x = x.relu()
     23       x = self.conv2(x, edge_index)

/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
   1100         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1101                 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1102             return forward_call(*input, **kwargs)
   1103         # Do not call functions when jit is used
   1104         full_backward_hooks, non_full_backward_hooks = [], []

/usr/local/lib/python3.8/dist-packages/torch_geometric/nn/conv/gcn_conv.py in forward(self, x, edge_index, edge_weight)
    180                     edge_index = cache
    181
--> 182         x = self.lin(x)
    183
    184         # propagate_type: (x: Tensor, edge_weight: OptTensor)

/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
   1100         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1101                 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1102             return forward_call(*input, **kwargs)
   1103         # Do not call functions when jit is used
   1104         full_backward_hooks, non_full_backward_hooks = [], []

/usr/local/lib/python3.8/dist-packages/torch_geometric/nn/dense/linear.py in forward(self, x)
    107     def forward(self, x: Tensor) -> Tensor:
    108         """"""
--> 109         return F.linear(x, self.weight, self.bias)
    110
    111     @torch.no_grad()

/usr/local/lib/python3.8/dist-packages/torch/nn/functional.py in linear(input, weight, bias)
   1846     if has_torch_function_variadic(input, weight, bias):
   1847         return handle_torch_function(linear, (input, weight, bias), input, weight, bias=bias)
-> 1848     return torch._C._nn.linear(input, weight, bias)
   1849
   1850

RuntimeError: expected scalar type Byte but found Float

Context:
The type mismatch is raised in:
out = model(data.pixel_values.view(-1, 3), data.edge_index, data.batch)

and based on the code snippet I guess that data.pixel_values might be returning a ByteTensor (uint8) while float32 is expected. Call .float() on this tensor, if that’s the case. If not, check which tensor is the ByteTensor and cast it to a FloatTensor.


Question:
I am trying to train GCN model on my custom dataset and I have resized all the values but I am getting error:

RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 7 but got size 14515200 for tensor number 1 in the list.

This is the GCN code:

class Net(torch.nn.Module):
def init(self):
super(Net, self).init()
self.conv1 = GCNConv(3, 64)
self.conv2 = GCNConv(64, 64)
self.conv3 = GCNConv(64, 64)
self.conv4 = GCNConv(48, 64)
self.conv5 = GCNConv(64, 96)
self.conv6 = GCNConv(96, 128)
self.linear1 = torch.nn.Linear(64,2)
self.linear2 = torch.nn.Linear(64,8)
def forward(self, data):
    print(data)
    pixel_values = data.pixel_values.view(-1, 3)
    x = torch.cat([data.pos, pixel_values], dim=-1)
    x, edge_index = x, data.edge_index
    x = self.conv1(x, edge_index)
    x = F.relu(x)
    x = self.conv2(x, edge_index)
    x = F.relu(x)
    x = self.conv3(x, edge_index)
    x = F.relu(x)
    x = self.conv4(x, edge_index)
    x = F.relu(x)
    x = self.conv5(x, edge_index)
    x = F.relu(x)
    x = self.conv6(x, edge_index)
    x = F.relu(x)
    x, _ = scatter_max(x, data.batch, dim=0)
    x = self.linear1(x)
    x = F.relu(x)
    x = self.linear2(x)
    return x

def main():
data_list = load_data_graph(data_size=data_size)
device = torch.device(‘cuda’)
model = Net().to(device)
trainset = data_list[:train_size]
optimizer = torch.optim.Adam(model.parameters())
train_loader = DataLoader(trainset, batch_size=batch_size, shuffle=True)
testset = data_list[train_size:]
test_loader = DataLoader(testset, batch_size=batch_size)
criterion = nn.CrossEntropyLoss()
history = {
    "train_loss": [],
    "test_loss": [],
    "test_acc": []
}

print("Start Train")

model.train()
for epoch in range(epoch_num):
    train_loss = 0.0
    for i, batch in enumerate(train_loader):
        batch = batch.to("cuda")
        optimizer.zero_grad()
        outputs = model(batch)
        loss = criterion(outputs,batch.tensor)
        loss.backward()
        optimizer.step()

        train_loss += loss.cpu().item()
        if i % 10 == 9:
            progress_bar = '['+('='*((i+1)//10))+(' '*((train_size//100-(i+1))//10))+']'
            print('\repoch: {:d} loss: {:.3f}  {}'
                    .format(epoch + 1, loss.cpu().item(), progress_bar), end="  ")

The error:

RuntimeError                              Traceback (most recent call last)
/tmp/ipykernel_58/874853799.py in
126
127 if name==“main”:
 → 128     main()
/tmp/ipykernel_58/874853799.py in main()
76             batch = batch.to(“cuda”)
77             optimizer.zero_grad()
—> 78             outputs = model(batch)
79             loss = criterion(outputs,batch.tensor)
80             loss.backward()
/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
1100         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
1101                 or _global_forward_hooks or _global_forward_pre_hooks):
 → 1102             return forward_call(*input, **kwargs)
1103         # Do not call functions when jit is used
1104         full_backward_hooks, non_full_backward_hooks = [], []
/tmp/ipykernel_58/874853799.py in forward(self, data)
28         print(data)
29         pixel_values = data.pixel_values.view(-1, 3)
—> 30         x = torch.cat([data.pos, pixel_values], dim=-1)
31         x, edge_index = x, data.edge_index
32         x = self.conv1(x, edge_index)
RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 7 but got size 14515200 for tensor number 1 in the list.

Context: torch.cat expects all dimension to have the same shape besides the one which is used to concatenate the tensors. Based on the error message you are hitting a shape mismatch in data.pos and pixel_values. Based on the code I guess the mismatch is caused in dim0 and you are trying to concatenate the tensors in dim1. This line of code looks related: pixel_values = data.pixel_values.view(-1, 3) as you are reshaping the pixel_values tensor right before this op so check what shape is expected.


Question:
I am trying to convert a list to tensor but I got an error.
ValueError: expected sequence of length 631 at dim 1 (got 284)

This code computes the pixel values for each detected object in the image.
imgs = PIL.Image.open(img_path).convert('RGB')
image_width, image_height = imgs.size
imgArrays = np.array(imgs)

X = (xCenter*image_width)
Y = (yCenter*image_height)
W = (Width*image_width)
H = (Height*image_height)

cropped_image = np.zeros((image_height, image_width, 3))

for i in range(len(X)):

           x1, y1, w, h = X[i], Y[i], W[i], H[i]

          x_start = int(x1 - (w/2))

          y_start = int(y1 - (h/2))

          x_end = int(x_start + w)

          y_end = int(y_start + h)

          temp = imgArrays[y_start: y_end, x_start: x_end]

          cropped_imagesList.append(temp)

          cropped_images = torch.as_tensor(cropped_imagesList)

Context:
The error sounds to me as if rows in the prospective tensor are not of the same length.
Tensors are strictly regular in this respect, if you want to process data of different lengths, you would need to resort to padding or something like that.
Best regards
Thomas


Is there any way to obtain the computation graph of an NN?

Question:
Hi,
I’d like to get the computation graph of a self-defined NN, is there any way in tensorboard that could help me to achieve that?

Context: The Tensorboard Tutorial 15 has a small example on how to visualize the model.


Question:
history = model.fit_generator(train_generator, epochs=epochs, steps_per_epoch=train_steps, verbose=1, callbacks=[checkpoint], validation_data=val_generator, validation_steps=val_steps)
def create_sequences(tokenizer, max_length, desc_list, photo, vocab_size):
X1, X2, y = list(), list(), list()
for desc in desc_list:
seq = tokenizer.texts_to_sequences([desc])[0]
for i in range(1, len(seq)):
in_seq, out_seq = seq[:i], seq[i] in_seq = pad_sequences([in_seq], maxlen=max_length)[0]
out_seq = to_categorical([out_seq], num_classes=vocab_size)[0]
X1.append(photo)
X2.append(in_seq)
y.append(out_seq)
return array(X1), array(X2), array(y)
def data_generator(descriptions, photos, tokenizer, max_length, imgsIds, vocab_size):
while 1:
for ind in range(len(imgsIds)):
photo = photos[ind]
key = imgsIds[ind]
desc_list = descriptions[str(key)]
in_img, in_seq, out_word = create_sequences( tokenizer, max_length, desc_list, photo, vocab_size) yield [in_img, in_seq], out_word

i got

82783/82783 [==============================] - ETA: 0s - loss: 4.0302Traceback (most recent call last):
File “rnn_flickr_fit.py”, line 281, in
main(sys.argv)
File “rnn_flickr_fit.py”, line 278, in main
fit_model(train, train_descriptions, train_rnn_input, val, val_descriptions, val_rnn_input)
File “rnn_flickr_fit.py”, line 265, in fit_model
history = model.fit(train_generator, epochs=epochs, steps_per_epoch=train_steps, verbose=1, callbacks=[checkpoint], validation_data=val_generator, validation_steps=val_steps)
File “/home/user/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py”, line 1213, in fit
steps_per_execution=self._steps_per_execution)
File “/home/user/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/data_adapter.py”, line 1364, in get_data_handler
return DataHandler(*args, **kwargs)
File “/home/user/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/data_adapter.py”, line 1166, in init
model=model)
File “/home/user/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/data_adapter.py”, line 811, in init
peek = _process_tensorlike(peek)
File “/home/user/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/data_adapter.py”, line 1046, in _process_tensorlike
inputs = nest.map_structure(_convert_numpy_and_scipy, inputs)
File “/home/user/.local/lib/python3.6/site-packages/tensorflow/python/util/nest.py”, line 867, in map_structure
structure[0], [func(*x) for x in entries],
File “/home/user/.local/lib/python3.6/site-packages/tensorflow/python/util/nest.py”, line 867, in
structure[0], [func(*x) for x in entries],
File “/home/user/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/data_adapter.py”, line 1041, in _convert_numpy_and_scipy
return ops.convert_to_tensor_v2_with_dispatch(x, dtype=dtype)
File “/home/user/.local/lib/python3.6/site-packages/tensorflow/python/util/dispatch.py”, line 206, in wrapper
return target(*args, **kwargs)
File “/home/user/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py”, line 1431, in convert_to_tensor_v2_with_dispatch
value, dtype=dtype, dtype_hint=dtype_hint, name=name)
File “/home/user/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py”, line 1441, in convert_to_tensor_v2
as_ref=False)
File “/home/user/.local/lib/python3.6/site-packages/tensorflow/python/profiler/trace.py”, line 163, in wrapped
return func(*args, **kwargs)
File “/home/user/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py”, line 1566, in convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File “/home/user/.local/lib/python3.6/site-packages/tensorflow/python/framework/tensor_conversion_registry.py”, line 52, in _default_conversion_function
return constant_op.constant(value, dtype, name=name)
File “/home/user/.local/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py”, line 265, in constant
allow_broadcast=True)
File “/home/user/.local/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py”, line 276, in _constant_impl
return _constant_eager_impl(ctx, value, dtype, shape, verify_shape)
File “/home/user/.local/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py”, line 301, in _constant_eager_impl
t = convert_to_eager_tensor(value, ctx, dtype)
File “/home/user/.local/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py”, line 98, in convert_to_eager_tensor
return ops.EagerTensor(value, ctx.device_name, dtype)
ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type dict).
2021-06-30 20:08:00.122545: W tensorflow/core/kernels/data/generator_dataset_op.cc:107] Error occurred when finalizing GeneratorDataset iterator: Failed precondition: Python interpreter state is not initialized. The process may be terminated.
[[{{node PyFunc}}]]

Context:
It seems you are using TensorFlow, so you might want to post the question in their discussion board or StackOverflow.
Based on the error message it seems that dicts are not supported as inputs to the conversion method.


Question:
Hey,
I’m wondering if it is possible in Tensorboard with pytorch to define groups of nodes that are displayed as one node. E.g. in TensorFlow it is possible to group and name nodes in Tensorboard using tf.name_scope (as far as I understood) as explained here 1. This is very helpful for having a better overview if you have several ‘units’ in your graph. A simple example, in a feed forward NN I could wish to group all the hidden layers in a node called ‘representation’ and the output layer in a node called ‘classifier’.
Is this somehow possible in pytorch? For me it seems like a very important feature yet I could’t find anything.
Cheers,
Spyderman

Context: Ok so I figured it out myself. You will get a node called ‘representation’ just by putting all your representation layers into an nn.sequential() with the variable name ‘representation’. So this is very intuitive. In fact, I already had it this way, I just didn’t recognize it because my model had so many parts that the graph was a bit messy.


How to add graphs to hparams in tensorboard?

Question:
I’m trying to use tensorboard with pytorch and cannot find any clear documentation as to how to add graphs to the hparams dashboard. The image below shows what I want
image1813×526 34.6 KB
however I can only add singular values for the metrics:
image1813×454 20.1 KB
Thanks in advance.

Context:
Having dug a little deeper into torch.utils.tensorboard, I can’t seem to find how to use this functionality. However I can achieve it by modifying the SummaryWriter.add_hparams() function as such:
			def add_hparams(self, hparam_dict, metric_dict, hparam_domain_discrete=None, run_name=None):
				torch._C._log_api_usage_once("tensorboard.logging.add_hparams")
				if type(hparam_dict) is not dict or type(metric_dict) is not dict:
					raise TypeError('hparam_dict and metric_dict should be dictionary.')
				exp, ssi, sei = hparams(hparam_dict, metric_dict, hparam_domain_discrete)

				self.file_writer.add_summary(exp)
				self.file_writer.add_summary(ssi)
				self.file_writer.add_summary(sei)
				for k, v in metric_dict.items():
					if v is not None:
						self.add_scalar(k, v)

To use this, when you create the SummaryWriter you should set log_dir=log_dir+'/[run_name]'. I have some example code and pictures to show how to use this below.
s = SummaryWriter(log_dir=f'{log_dir}/run1')
h_params = {'This': 1, 'is': 2, 'a': 3, 'test': 4}
metrics = {'accuracy/accuracy': None, 'loss/loss': None}

accuracy_values = [(1, 0.6), (2, 0.8), (3, 0.9), (4, 0.95)]
loss_values = [(1, 3), (2, 1), (3, 0.5), (4, 0.11)]
for step, v in accuracy_values:
	s.add_scalar('accuracy/accuracy', v, step)
for step, v in loss_values:
	s.add_scalar('loss/loss', v, step)

s.add_hparams(h_params, metrics)
s.close()

image1813×454 30.7 KB
Something that I’m not sure if it’s a bug or not is that in the original code, any scalars are logged to the log_dir whilst hyper parameters are logged in their own directory and given a run_name.
(I’ve just posted this as hopefully this is useful to someone)


Question:
Hi everyone,
I want to make a tensor T of dim n (arbitrary)  which is something like T_{i1,...,in} such that each of i can be 0, 1, or 2 which must be something like  T = torch.tensor(3, 3, ..., 3). Do you have any idea how can I do this?
Many thanks in advance.

Context:
I don’t fully understand the use case.
Would you like to set the size as 0, 1, or 2 or the input values?
The former case would be possible via the factory methods, e.g.: torch.randn(3), while the latter would work via e.g. tensor.fill_(value) or torch.full(...).


All experiments were written as one (without provided tags)

Question:
Hi, I wanted to compare several runs that I did in the loop creating new SummaryWriter instances like this:
for experiment_name in experiments:
    logger = SummaryWriter(self._log_path, comment=experiment_name)
    ...
    for epoch in range(5):
        ...
        logger.add_scalar("Epoch Loss", loss, epoch)
        ...
    logger.close()

In the log path I got several files like this:
events.out.tfevents.1609675249.nlp-vm.13735.0
events.out.tfevents.1609679736.nlp-vm.13735.1
events.out.tfevents.1609687200.nlp-vm.13735.2
events.out.tfevents.1609691662.nlp-vm.13735.3
events.out.tfevents.1609699158.nlp-vm.13735.4
events.out.tfevents.1609703743.nlp-vm.13735.5
events.out.tfevents.1609711308.nlp-vm.13735.6
events.out.tfevents.1609716054.nlp-vm.13735.7

But the Tensorboard displays all runs as one:

What I should do to fix it and can I do it without re-running all experiments?

Context:
Found: the events files that I have in the one folder should be in the separate and the folder name will be displayed as an experiment name.
Also found the important note in the SummaryWriter docstring:
comment (string): Comment log_dir suffix appended to the default
``log_dir``. If ``log_dir`` is assigned, this argument has no effect.


Is it possible to rename Event Log scalar

Question:
Hi,
I’ve been fairly busy running and updating several pieces of code as I learn pytorch.
After a bunch of experiments that took a fair amount of time I noticed that some of my logged scalars on tensorboard had different names/paths. I’d like to set them under the same name/path, offline, instead of re-running all experiments just to get new logs with the issue fixed.
Is there any possibility of doing this in tensorboard or tensorboardX ? I can’t seem to find much information online about this, nor the documentation seems to have any functions for renaming stuff inside already finished logs.
Thanks in advance.

Context:
Well after a while I finaly found something that proved useful:

      stackoverflow.com

  How do you edit an existing Tensorboard Training Loss summary? 169

  python, tensorflow, machine-learning, edit

  asked by

    AlexP

  on 04:01PM - 05 Feb 20 UTC

The accepted answer managed to work for my case. Although I  have to keep calling the main function and go about making a new output event file per scalar to change. I could probably change it to work for multiple pairs of oldnames/newnames but as it is works well enough and is made to replace multiple old names to a single new name.


How to add activation histogram in tensorboard?

Question:
Currently this is how i add a histogram for my layer’s weights:
writer.add_histogram(f'Classifier/p/Weights',model.fc[-1].weight, epoch)

How can I add the activation histogram in a similar manner?
My assumption would be that I have to add it in the forward function after it passes through the relevant ReLU, softmax, etc. However what do I do when i don’t have access to the forward function directly (model imported from torchvision.models, or defined as a nn.Sequential).
Example of what i’m looking for taken from tensorboard.dev 4:

Context: You can use forward hooks to get the internal activation as described here 97.


Torch.utils.tensorboard fails with multiple inputs model

Question:
Hi all! I have a matching network model which receives 4 inputs, while it all works  correctly, once I add in the tensorboard add_graph functioni n the mix it fails miserably.
This is how the model is called:
acc, c_loss, preds = model(x_support_set, y_support_set_one_hot, x_target, y_target)
And this is the add_graph call:
writer.add_graph(model, (x_support_set, y_support_set_one_hot, x_target, y_target), True)
The error is from an internal method in the model where I encode the inputs.
Full trace here:
/home/davide/Desktop/coefficient/native_oneshot/native_oneshot/matching_network/model/matching.py:90: TracerWarning: Converting a tensor to a Python index might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  batch_size = len(target_images)
/home/davide/Desktop/coefficient/native_oneshot/native_oneshot/matching_network/model/matching.py:55: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  print(support_set_images.size(), target_images.size())
torch.Size([1, 25, 3, 100, 100]) torch.Size([10, 3, 100, 100])
/home/davide/Desktop/coefficient/native_oneshot/native_oneshot/matching_network/model/matching.py:56: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  for i in np.arange(support_set_images.size(1)):
Integer division of tensors using div or / is no longer supported, and in a future release div will perform true division as in Python 3. Use true_divide or floor_divide (// in Python) instead.
Error occurs, No graph saved
 96%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▍         | 24/25 [00:06<00:00,  3.85it/s]
Traceback (most recent call last):
  File "/home/davide/.pyenv/versions/3.7.8/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/davide/.pyenv/versions/3.7.8/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/davide/Desktop/coefficient/native_oneshot/native_oneshot/scripts/matchnet_train.py", line 298, in <module>
    log_dir=model_folder_path,
  File "/home/davide/Desktop/coefficient/native_oneshot/native_oneshot/scripts/matchnet_train.py", line 203, in run_epoch
    batch_number=i,
  File "/home/davide/Desktop/coefficient/native_oneshot/native_oneshot/scripts/matchnet_train.py", line 174, in run_batch
    True,
  File "/home/davide/.virtualenvs/native/lib/python3.7/site-packages/torch/utils/tensorboard/writer.py", line 714, in add_graph
    self._get_file_writer().add_graph(graph(model, input_to_model, verbose))
  File "/home/davide/.virtualenvs/native/lib/python3.7/site-packages/torch/utils/tensorboard/_pytorch_graph.py", line 291, in graph
    raise e
  File "/home/davide/.virtualenvs/native/lib/python3.7/site-packages/torch/utils/tensorboard/_pytorch_graph.py", line 285, in graph
    trace = torch.jit.trace(model, args)
  File "/home/davide/.virtualenvs/native/lib/python3.7/site-packages/torch/jit/__init__.py", line 955, in trace
    check_tolerance, strict, _force_outplace, _module_class)
  File "/home/davide/.virtualenvs/native/lib/python3.7/site-packages/torch/jit/__init__.py", line 1109, in trace_module
    module._c._create_method_from_trace(method_name, func, example_inputs, var_lookup_fn, strict, _force_outplace)
  File "/home/davide/.virtualenvs/native/lib/python3.7/site-packages/torch/nn/modules/module.py", line 720, in _call_impl
    result = self._slow_forward(*input, **kwargs)
  File "/home/davide/.virtualenvs/native/lib/python3.7/site-packages/torch/nn/modules/module.py", line 704, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "/home/davide/Desktop/coefficient/native_oneshot/native_oneshot/matching_network/model/matching.py", line 77, in forward
    similarities = self.predict_similarities(support_set_images, target_images)
  File "/home/davide/Desktop/coefficient/native_oneshot/native_oneshot/matching_network/model/matching.py", line 91, in predict_similarities
    encodings = self.encode_support_and_target(support_set_images, target_images)
  File "/home/davide/Desktop/coefficient/native_oneshot/native_oneshot/matching_network/model/matching.py", line 56, in encode_support_and_target
    for i in np.arange(support_set_images.size(1)):
RuntimeError: Integer division of tensors using div or / is no longer supported, and in a future release div will perform true division as in Python 3. Use true_divide or floor_divide (// in Python) instead.

specifically the exception is thrown on this line for i in np.arange(support_set_images.size(1)): where support_set_images is of size torch.Size([1, 25, 3, 100, 100])
I’m a bit lost as to why everything would train fine, and when adding the add_graph call would throw such an exception deeper in the code, while the correct data is being passed. Any help would be appreciated!

Context: As a side note, I already verified that the tensors are both FloatTensors as per this thread RuntimeError: Integer division of tensors using div or / is no longer supported, and in a future release div will perform true division as in Python 3. Use true_divide or floor_divide (// in Python) instead 7


Question:
I am trying to figure out how to go about uninstalling Python 3.10. on my computer which is a Mac OS Big Sur laptop (Version 11.6.2).
I am ultimately trying to install PyTorch on my laptop, but I was having trouble trying to install it via Python 3.10.
Would it be advisable to uninstall Python 3.10 on my computer by going to the terminal and entering the following command?
sudo rm -rf /Library/Frameworks/Python.framework/Versions/3.10

Context: Maybe a better approach would be to create a new virtual environment (e.g. via conda) and install Python=3.9 there while leaving the system 3.10 installation intact.