Repository: facebookresearch/habitat-lab

Issue:
TypeError: can't assign a numpy.ndarray to a torch.ByteTensor
## Habitat-Lab and Habitat-Sim versions
habitat                 0.2.1              
habitat-sim             0.2.1

## ❓ Questions and Help
### I have questions when I run the [habitat baseline](https://github.com/facebookresearch/habitat-lab/tree/master/habitat_baselines), after I download the [test dataset ](http://dl.fbaipublicfiles.com/habitat/habitat-test-scenes.zip) and unzip it correctly and I run this command:
`python -u habitat_baselines/run.py --exp-config habitat_baselines/config/pointnav/ppo_pointnav_example.yaml --run-type train
`  

then the problem occurs, all of the information are as follows:
```
(habitat) sdu310@sdu310-Precision-5820-Tower-X-Series:~/habitat-lab$ python -u habitat_baselines/run.py --exp-config habitat_baselines/config/pointnav/ppo_pointnav_example.yaml --run-type train > ../jyj_personal/train
/home/sdu310/anaconda3/envs/habitat/lib/python3.6/site-packages/torch-1.9.1-py3.6-linux-x86_64.egg/torch/cuda/__init__.py:52: UserWarning: CUDA initialization: The NVIDIA driver on your system is too old (found version 10010). Please update your GPU driver by downloading and installing a new version from the URL: http://www.nvidia.com/Download/index.aspx Alternatively, go to: https://pytorch.org to install a PyTorch version that has been compiled with your version of the CUDA driver. (Triggered internally at  ../c10/cuda/CUDAFunctions.cpp:115.)
  return torch._C._cuda_getDeviceCount() > 0
2021-10-01 20:37:10,095 config: BASE_TASK_CONFIG_PATH: configs/tasks/pointnav.yaml
CHECKPOINT_FOLDER: data/new_checkpoints
CHECKPOINT_INTERVAL: -1
CMD_TRAILING_OPTS: []
ENV_NAME: NavRLEnv
EVAL:
  SPLIT: val
  USE_CKPT_CONFIG: True
EVAL_CKPT_PATH_DIR: data/new_checkpoints
FORCE_BLIND_POLICY: False
FORCE_TORCH_SINGLE_THREADED: True
LOG_FILE: train.log
LOG_INTERVAL: 10
NUM_CHECKPOINTS: 50
NUM_ENVIRONMENTS: 1
NUM_PROCESSES: -1
NUM_UPDATES: -1
ORBSLAM2:
  ANGLE_TH: 0.2617993877991494
  BETA: 100
  CAMERA_HEIGHT: 1.25
  DEPTH_DENORM: 10.0
  DIST_REACHED_TH: 0.15
  DIST_TO_STOP: 0.05
  D_OBSTACLE_MAX: 4.0
  D_OBSTACLE_MIN: 0.1
  H_OBSTACLE_MAX: 1.25
  H_OBSTACLE_MIN: 0.375
  MAP_CELL_SIZE: 0.1
  MAP_SIZE: 40
  MIN_PTS_IN_OBSTACLE: 320.0
  NEXT_WAYPOINT_TH: 0.5
  NUM_ACTIONS: 3
  PLANNER_MAX_STEPS: 500
  PREPROCESS_MAP: True
  SLAM_SETTINGS_PATH: habitat_baselines/slambased/data/mp3d3_small1k.yaml
  SLAM_VOCAB_PATH: habitat_baselines/slambased/data/ORBvoc.txt
PROFILING:
  CAPTURE_START_STEP: -1
  NUM_STEPS_TO_CAPTURE: -1
RL:
  DDPPO:
    backbone: resnet18
    distrib_backend: GLOO
    force_distributed: False
    num_recurrent_layers: 1
    pretrained: False
    pretrained_encoder: False
    pretrained_weights: data/ddppo-models/gibson-2plus-resnet50.pth
    reset_critic: True
    rnn_type: GRU
    sync_frac: 0.6
    train_encoder: True
  POLICY:
    ACTION_DIST:
      action_activation: tanh
      max_log_std: 2
      max_std: 1
      min_log_std: -5
      min_std: 1e-06
      use_log_std: False
      use_softplus: False
    OBS_TRANSFORMS:
      CENTER_CROPPER:
        HEIGHT: 256
        WIDTH: 256
      CUBE2EQ:
        HEIGHT: 256
        SENSOR_UUIDS: []
        WIDTH: 512
      CUBE2FISH:
        FOV: 180
        HEIGHT: 256
        PARAMS: (0.2, 0.2, 0.2)
        SENSOR_UUIDS: []
        WIDTH: 256
      ENABLED_TRANSFORMS: ()
      EQ2CUBE:
        HEIGHT: 256
        SENSOR_UUIDS: []
        WIDTH: 256
      RESIZE_SHORTEST_EDGE:
        SIZE: 256
    action_distribution_type: categorical
    name: PointNavResNetPolicy
  PPO:
    clip_param: 0.1
    entropy_coef: 0.01
    eps: 1e-05
    gamma: 0.99
    hidden_size: 512
    lr: 0.00025
    max_grad_norm: 0.5
    num_mini_batch: 1
    num_steps: 32
    ppo_epoch: 1
    reward_window_size: 50
    tau: 0.95
    use_double_buffered_sampler: False
    use_gae: True
    use_linear_clip_decay: True
    use_linear_lr_decay: True
    use_normalized_advantage: False
    value_loss_coef: 0.5
  REWARD_MEASURE: distance_to_goal
  SLACK_REWARD: -0.01
  SUCCESS_MEASURE: spl
  SUCCESS_REWARD: 2.5
  preemption:
    append_slurm_job_id: False
    save_resume_state_interval: 100
    save_state_batch_only: False
SENSORS: ['RGB_SENSOR', 'DEPTH_SENSOR']
SIMULATOR_GPU_ID: 0
TASK_CONFIG:
  DATASET:
    CONTENT_SCENES: ['*']
    DATA_PATH: data/datasets/pointnav/habitat-test-scenes/v1/{split}/{split}.json.gz
    SCENES_DIR: data/scene_datasets
    SPLIT: train
    TYPE: PointNav-v1
  ENVIRONMENT:
    ITERATOR_OPTIONS:
      CYCLE: True
      GROUP_BY_SCENE: True
      MAX_SCENE_REPEAT_EPISODES: -1
      MAX_SCENE_REPEAT_STEPS: 10000
      NUM_EPISODE_SAMPLE: -1
      SHUFFLE: True
      STEP_REPETITION_RANGE: 0.2
    MAX_EPISODE_SECONDS: 10000000
    MAX_EPISODE_STEPS: 500
  PYROBOT:
    BASE_CONTROLLER: proportional
    BASE_PLANNER: none
    BUMP_SENSOR:
      TYPE: PyRobotBumpSensor
    DEPTH_SENSOR:
      CENTER_CROP: False
      HEIGHT: 480
      MAX_DEPTH: 5.0
      MIN_DEPTH: 0.0
      NORMALIZE_DEPTH: True
      TYPE: PyRobotDepthSensor
      WIDTH: 640
    LOCOBOT:
      ACTIONS: ['BASE_ACTIONS', 'CAMERA_ACTIONS']
      BASE_ACTIONS: ['go_to_relative', 'go_to_absolute']
      CAMERA_ACTIONS: ['set_pan', 'set_tilt', 'set_pan_tilt']
    RGB_SENSOR:
      CENTER_CROP: False
      HEIGHT: 480
      TYPE: PyRobotRGBSensor
      WIDTH: 640
    ROBOT: locobot
    ROBOTS: ['locobot']
    SENSORS: ['RGB_SENSOR', 'DEPTH_SENSOR', 'BUMP_SENSOR']
  SEED: 100
  SIMULATOR:
    ACTION_SPACE_CONFIG: v0
    AGENTS: ['AGENT_0']
    AGENT_0:
      ANGULAR_ACCELERATION: 12.56
      ANGULAR_FRICTION: 1.0
      COEFFICIENT_OF_RESTITUTION: 0.0
      HEIGHT: 1.5
      IS_SET_START_STATE: False
      LINEAR_ACCELERATION: 20.0
      LINEAR_FRICTION: 0.5
      MASS: 32.0
      RADIUS: 0.1
      SENSORS: ['RGB_SENSOR']
      START_POSITION: [0, 0, 0]
      START_ROTATION: [0, 0, 0, 1]
    ARM_DEPTH_SENSOR:
      HEIGHT: 480
      HFOV: 90
      MAX_DEPTH: 10.0
      MIN_DEPTH: 0.0
      NORMALIZE_DEPTH: True
      ORIENTATION: [0.0, 0.0, 0.0]
      POSITION: [0, 1.25, 0]
      SENSOR_SUBTYPE: PINHOLE
      TYPE: HabitatSimDepthSensor
      UUID: robot_arm_depth
      WIDTH: 640
    ARM_RGB_SENSOR:
      HEIGHT: 480
      HFOV: 90
      ORIENTATION: [0.0, 0.0, 0.0]
      POSITION: [0, 1.25, 0]
      SENSOR_SUBTYPE: PINHOLE
      TYPE: HabitatSimRGBSensor
      UUID: robot_arm_rgb
      WIDTH: 640
    DEFAULT_AGENT_ID: 0
    DEPTH_SENSOR:
      HEIGHT: 256
      HFOV: 90
      MAX_DEPTH: 10.0
      MIN_DEPTH: 0.0
      NORMALIZE_DEPTH: True
      ORIENTATION: [0.0, 0.0, 0.0]
      POSITION: [0, 1.25, 0]
      SENSOR_SUBTYPE: PINHOLE
      TYPE: HabitatSimDepthSensor
      WIDTH: 256
    EQUIRECT_DEPTH_SENSOR:
      HEIGHT: 480
      MAX_DEPTH: 10.0
      MIN_DEPTH: 0.0
      NORMALIZE_DEPTH: True
      ORIENTATION: [0.0, 0.0, 0.0]
      POSITION: [0, 1.25, 0]
      TYPE: HabitatSimEquirectangularDepthSensor
      WIDTH: 640
    EQUIRECT_RGB_SENSOR:
      HEIGHT: 480
      ORIENTATION: [0.0, 0.0, 0.0]
      POSITION: [0, 1.25, 0]
      TYPE: HabitatSimEquirectangularRGBSensor
      WIDTH: 640
    EQUIRECT_SEMANTIC_SENSOR:
      HEIGHT: 480
      ORIENTATION: [0.0, 0.0, 0.0]
      POSITION: [0, 1.25, 0]
      TYPE: HabitatSimEquirectangularSemanticSensor
      WIDTH: 640
    FISHEYE_DEPTH_SENSOR:
      ALPHA: 0.57
      FOCAL_LENGTH: [364.84, 364.86]
      HEIGHT: 480
      MAX_DEPTH: 10.0
      MIN_DEPTH: 0.0
      NORMALIZE_DEPTH: True
      ORIENTATION: [0.0, 0.0, 0.0]
      POSITION: [0, 1.25, 0]
      PRINCIPAL_POINT_OFFSET: None
      SENSOR_MODEL_TYPE: DOUBLE_SPHERE
      TYPE: HabitatSimFisheyeDepthSensor
      WIDTH: 640
      XI: -0.27
    FISHEYE_RGB_SENSOR:
      ALPHA: 0.57
      FOCAL_LENGTH: [364.84, 364.86]
      HEIGHT: 640
      ORIENTATION: [0.0, 0.0, 0.0]
      POSITION: [0, 1.25, 0]
      PRINCIPAL_POINT_OFFSET: None
      SENSOR_MODEL_TYPE: DOUBLE_SPHERE
      TYPE: HabitatSimFisheyeRGBSensor
      WIDTH: 640
      XI: -0.27
    FISHEYE_SEMANTIC_SENSOR:
      ALPHA: 0.57
      FOCAL_LENGTH: [364.84, 364.86]
      HEIGHT: 640
      ORIENTATION: [0.0, 0.0, 0.0]
      POSITION: [0, 1.25, 0]
      PRINCIPAL_POINT_OFFSET: None
      SENSOR_MODEL_TYPE: DOUBLE_SPHERE
      TYPE: HabitatSimFisheyeSemanticSensor
      WIDTH: 640
      XI: -0.27
    FORWARD_STEP_SIZE: 0.25
    HABITAT_SIM_V0:
      ALLOW_SLIDING: True
      ENABLE_PHYSICS: False
      GPU_DEVICE_ID: 0
      GPU_GPU: False
      PHYSICS_CONFIG_FILE: ./data/default.physics_config.json
    HEAD_DEPTH_SENSOR:
      HEIGHT: 480
      HFOV: 90
      MAX_DEPTH: 10.0
      MIN_DEPTH: 0.0
      NORMALIZE_DEPTH: True
      ORIENTATION: [0.0, 0.0, 0.0]
      POSITION: [0, 1.25, 0]
      SENSOR_SUBTYPE: PINHOLE
      TYPE: HabitatSimDepthSensor
      UUID: robot_head_depth
      WIDTH: 640
    HEAD_RGB_SENSOR:
      HEIGHT: 480
      HFOV: 90
      ORIENTATION: [0.0, 0.0, 0.0]
      POSITION: [0, 1.25, 0]
      SENSOR_SUBTYPE: PINHOLE
      TYPE: HabitatSimRGBSensor
      UUID: robot_head_rgb
      WIDTH: 640
    RGB_SENSOR:
      HEIGHT: 256
      HFOV: 90
      ORIENTATION: [0.0, 0.0, 0.0]
      POSITION: [0, 1.25, 0]
      SENSOR_SUBTYPE: PINHOLE
      TYPE: HabitatSimRGBSensor
      WIDTH: 256
    SCENE: data/scene_datasets/habitat-test-scenes/van-gogh-room.glb
    SEED: 100
    SEMANTIC_SENSOR:
      HEIGHT: 480
      HFOV: 90
      ORIENTATION: [0.0, 0.0, 0.0]
      POSITION: [0, 1.25, 0]
      SENSOR_SUBTYPE: PINHOLE
      TYPE: HabitatSimSemanticSensor
      WIDTH: 640
    THIRD_DEPTH_SENSOR:
      HEIGHT: 480
      HFOV: 90
      MAX_DEPTH: 10.0
      MIN_DEPTH: 0.0
      NORMALIZE_DEPTH: True
      ORIENTATION: [0.0, 0.0, 0.0]
      POSITION: [0, 1.25, 0]
      SENSOR_SUBTYPE: PINHOLE
      TYPE: HabitatSimDepthSensor
      UUID: robot_third_rgb
      WIDTH: 640
    THIRD_RGB_SENSOR:
      HEIGHT: 480
      HFOV: 90
      ORIENTATION: [0.0, 0.0, 0.0]
      POSITION: [0, 1.25, 0]
      SENSOR_SUBTYPE: PINHOLE
      TYPE: HabitatSimRGBSensor
      UUID: robot_third_rgb
      WIDTH: 640
    TILT_ANGLE: 15
    TURN_ANGLE: 10
    TYPE: Sim-v0
  TASK:
    ACTIONS:
      ANSWER:
        TYPE: AnswerAction
      LOOK_DOWN:
        TYPE: LookDownAction
      LOOK_UP:
        TYPE: LookUpAction
      MOVE_FORWARD:
        TYPE: MoveForwardAction
      STOP:
        TYPE: StopAction
      TELEPORT:
        TYPE: TeleportAction
      TURN_LEFT:
        TYPE: TurnLeftAction
      TURN_RIGHT:
        TYPE: TurnRightAction
      VELOCITY_CONTROL:
        ANG_VEL_RANGE: [-10.0, 10.0]
        LIN_VEL_RANGE: [0.0, 0.25]
        MIN_ABS_ANG_SPEED: 1.0
        MIN_ABS_LIN_SPEED: 0.025
        TIME_STEP: 1.0
        TYPE: VelocityAction
    ANSWER_ACCURACY:
      TYPE: AnswerAccuracy
    COLLISIONS:
      TYPE: Collisions
    COMPASS_SENSOR:
      TYPE: CompassSensor
    CORRECT_ANSWER:
      TYPE: CorrectAnswer
    DISTANCE_TO_GOAL:
      DISTANCE_TO: POINT
      TYPE: DistanceToGoal
    EPISODE_INFO:
      TYPE: EpisodeInfo
    GOAL_SENSOR_UUID: pointgoal_with_gps_compass
    GPS_SENSOR:
      DIMENSIONALITY: 2
      TYPE: GPSSensor
    HEADING_SENSOR:
      TYPE: HeadingSensor
    IMAGEGOAL_SENSOR:
      TYPE: ImageGoalSensor
    INSTRUCTION_SENSOR:
      TYPE: InstructionSensor
    INSTRUCTION_SENSOR_UUID: instruction
    MEASUREMENTS: ['DISTANCE_TO_GOAL', 'SUCCESS', 'SPL']
    OBJECTGOAL_SENSOR:
      GOAL_SPEC: TASK_CATEGORY_ID
      GOAL_SPEC_MAX_VAL: 50
      TYPE: ObjectGoalSensor
    POINTGOAL_SENSOR:
      DIMENSIONALITY: 2
      GOAL_FORMAT: POLAR
      TYPE: PointGoalSensor
    POINTGOAL_WITH_GPS_COMPASS_SENSOR:
      DIMENSIONALITY: 2
      GOAL_FORMAT: POLAR
      TYPE: PointGoalWithGPSCompassSensor
    POSSIBLE_ACTIONS: ['STOP', 'MOVE_FORWARD', 'TURN_LEFT', 'TURN_RIGHT']
    PROXIMITY_SENSOR:
      MAX_DETECTION_RADIUS: 2.0
      TYPE: ProximitySensor
    QUESTION_SENSOR:
      TYPE: QuestionSensor
    SENSORS: ['POINTGOAL_WITH_GPS_COMPASS_SENSOR']
    SOFT_SPL:
      TYPE: SoftSPL
    SPL:
      TYPE: SPL
    SUCCESS:
      SUCCESS_DISTANCE: 0.2
      TYPE: Success
    SUCCESS_DISTANCE: 0.2
    TOP_DOWN_MAP:
      DRAW_BORDER: True
      DRAW_GOAL_AABBS: True
      DRAW_GOAL_POSITIONS: True
      DRAW_SHORTEST_PATH: True
      DRAW_SOURCE: True
      DRAW_VIEW_POINTS: True
      FOG_OF_WAR:
        DRAW: True
        FOV: 90
        VISIBILITY_DIST: 5.0
      MAP_PADDING: 3
      MAP_RESOLUTION: 1024
      MAX_EPISODE_STEPS: 1000
      TYPE: TopDownMap
    TYPE: Nav-v0
TENSORBOARD_DIR: tb
TEST_EPISODE_COUNT: 2
TORCH_GPU_ID: 0
TOTAL_NUM_STEPS: 1000000.0
TRAINER_NAME: ppo
VERBOSE: True
VIDEO_DIR: video_dir
VIDEO_OPTION: ['disk', 'tensorboard']
2021-10-01 20:37:10,095 Initializing dataset PointNav-v1
2021-10-01 20:37:12,035 Initializing dataset PointNav-v1
2021-10-01 20:37:12,223 initializing sim Sim-v0
WARNING: Logging before InitGoogleLogging() is written to STDERR
I1001 20:37:12.224717 20600 ManagedFileBasedContainer.h:210] <Dataset>::convertFilenameToPassedExt : Filename : default changed to proposed scene_dataset_config.json filename : default.scene_dataset_config.json
I1001 20:37:12.224750 20600 AttributesManagerBase.h:365] <Dataset>::createFromJsonOrDefaultInternal : Proposing JSON name : default.scene_dataset_config.json from original name : default | This file  does not exist.
I1001 20:37:12.224823 20600 AssetAttributesManager.cpp:120] Asset attributes (capsule3DSolid : capsule3DSolid_hemiRings_4_cylRings_1_segments_12_halfLen_0.75_useTexCoords_false_useTangents_false) created and registered.
I1001 20:37:12.224861 20600 AssetAttributesManager.cpp:120] Asset attributes (capsule3DWireframe : capsule3DWireframe_hemiRings_8_cylRings_1_segments_16_halfLen_1) created and registered.
I1001 20:37:12.224896 20600 AssetAttributesManager.cpp:120] Asset attributes (coneSolid : coneSolid_segments_12_halfLen_1.25_rings_1_useTexCoords_false_useTangents_false_capEnd_true) created and registered.
I1001 20:37:12.224921 20600 AssetAttributesManager.cpp:120] Asset attributes (coneWireframe : coneWireframe_segments_32_halfLen_1.25) created and registered.
I1001 20:37:12.224937 20600 AssetAttributesManager.cpp:120] Asset attributes (cubeSolid : cubeSolid) created and registered.
I1001 20:37:12.224949 20600 AssetAttributesManager.cpp:120] Asset attributes (cubeWireframe : cubeWireframe) created and registered.
I1001 20:37:12.224979 20600 AssetAttributesManager.cpp:120] Asset attributes (cylinderSolid : cylinderSolid_rings_1_segments_12_halfLen_1_useTexCoords_false_useTangents_false_capEnds_true) created and registered.
I1001 20:37:12.225006 20600 AssetAttributesManager.cpp:120] Asset attributes (cylinderWireframe : cylinderWireframe_rings_1_segments_32_halfLen_1) created and registered.
I1001 20:37:12.225023 20600 AssetAttributesManager.cpp:120] Asset attributes (icosphereSolid : icosphereSolid_subdivs_1) created and registered.
I1001 20:37:12.225036 20600 AssetAttributesManager.cpp:120] Asset attributes (icosphereWireframe : icosphereWireframe_subdivs_1) created and registered.
I1001 20:37:12.225055 20600 AssetAttributesManager.cpp:120] Asset attributes (uvSphereSolid : uvSphereSolid_rings_8_segments_16_useTexCoords_false_useTangents_false) created and registered.
I1001 20:37:12.225075 20600 AssetAttributesManager.cpp:120] Asset attributes (uvSphereWireframe : uvSphereWireframe_rings_16_segments_32) created and registered.
I1001 20:37:12.225083 20600 AssetAttributesManager.cpp:108] ::constructor : Built default primitive asset templates : 12
I1001 20:37:12.225602 20600 SceneDatasetAttributesManager.cpp:36] File (default) not found, so new default dataset attributes created and registered.
I1001 20:37:12.225608 20600 MetadataMediator.cpp:127] ::createSceneDataset : Dataset default successfully created.
I1001 20:37:12.225616 20600 AttributesManagerBase.h:365] <Physics Manager>::createFromJsonOrDefaultInternal : Proposing JSON name : ./data/default.physics_config.json from original name : ./data/default.physics_config.json | This file  does not exist.
I1001 20:37:12.225636 20600 PhysicsAttributesManager.cpp:26] File (./data/default.physics_config.json) not found, so new default physics manager attributes created and registered.
I1001 20:37:12.225641 20600 MetadataMediator.cpp:212] ::setActiveSceneDatasetName : Previous active dataset  changed to default successfully.
I1001 20:37:12.225646 20600 MetadataMediator.cpp:183] ::setCurrPhysicsAttributesHandle : Old physics manager attributes  changed to ./data/default.physics_config.json successfully.
I1001 20:37:12.225651 20600 MetadataMediator.cpp:68] ::setSimulatorConfiguration : Set new simulator config for scene/stage : data/scene_datasets/habitat-test-scenes/skokloster-castle.glb and dataset : default which is currently active dataset.
I1001 20:37:12.294502 20600 ManagedFileBasedContainer.h:210] <Scene Instance>::convertFilenameToPassedExt : Filename : data/scene_datasets/habitat-test-scenes/skokloster-castle.glb changed to proposed scene_instance.json filename : data/scene_datasets/habitat-test-scenes/skokloster-castle.scene_instance.json
I1001 20:37:12.294533 20600 MetadataMediator.cpp:311] ::getSceneAttributesByName : Dataset : default has no preloaded SceneAttributes or StageAttributes named : data/scene_datasets/habitat-test-scenes/skokloster-castle.glb so loading/creating a new StageAttributes with this name, and then creating a SceneAttributes with the same name that references this stage.
I1001 20:37:12.294548 20600 ManagedFileBasedContainer.h:210] <Stage Template>::convertFilenameToPassedExt : Filename : data/scene_datasets/habitat-test-scenes/skokloster-castle.glb changed to proposed stage_config.json filename : data/scene_datasets/habitat-test-scenes/skokloster-castle.stage_config.json
I1001 20:37:12.294559 20600 AttributesManagerBase.h:365] <Stage Template>::createFromJsonOrDefaultInternal : Proposing JSON name : data/scene_datasets/habitat-test-scenes/skokloster-castle.stage_config.json from original name : data/scene_datasets/habitat-test-scenes/skokloster-castle.glb | This file  does not exist.
I1001 20:37:12.294697 20600 AbstractObjectAttributesManagerBase.h:183] File (data/scene_datasets/habitat-test-scenes/skokloster-castle.glb) exists but is not a recognized config filename extension, so new default Stage Template attributes created and registered.
I1001 20:37:12.294751 20600 SceneDatasetAttributes.cpp:45] ::addNewSceneInstanceToDataset : Dataset : 'default' : Stage Attributes 'data/scene_datasets/habitat-test-scenes/skokloster-castle.glb' specified in Scene Attributes exists in dataset library.
I1001 20:37:12.294760 20600 SceneDatasetAttributes.cpp:79] ::addNewSceneInstanceToDataset : Dataset : 'default' : Lighting Layout Attributes 'no_lights' specified in Scene Attributes but does not exist in dataset, so creating.
I1001 20:37:12.294770 20600 ManagedFileBasedContainer.h:210] <Lighting Layout>::convertFilenameToPassedExt : Filename : no_lights changed to proposed lighting_config.json filename : no_lights.lighting_config.json
I1001 20:37:12.294782 20600 ManagedFileBasedContainer.h:210] <Lighting Layout>::convertFilenameToPassedExt : Filename : no_lights changed to proposed lighting_config.json filename : no_lights.lighting_config.json
I1001 20:37:12.294791 20600 AttributesManagerBase.h:365] <Lighting Layout>::createFromJsonOrDefaultInternal : Proposing JSON name : no_lights.lighting_config.json from original name : no_lights | This file  does not exist.
I1001 20:37:12.294816 20600 LightLayoutAttributesManager.cpp:34] File (no_lights) not found, so new default light layout attributes created and registered.
I1001 20:37:12.294827 20600 Simulator.cpp:206] ::setSceneInstanceAttributes : Navmesh file location in scene instance : data/scene_datasets/habitat-test-scenes/skokloster-castle.navmesh
I1001 20:37:12.294838 20600 Simulator.cpp:213] ::setSceneInstanceAttributes : Loading navmesh from data/scene_datasets/habitat-test-scenes/skokloster-castle.navmesh
I1001 20:37:12.294989 20600 Simulator.cpp:216] ::setSceneInstanceAttributes : Navmesh Loaded.
I1001 20:37:12.295008 20600 SceneGraph.h:85] Created DrawableGroup: 
I1001 20:37:12.295017 20600 Simulator.cpp:251] ::setSceneInstanceAttributes : SceneInstance : data/scene_datasets/habitat-test-scenes/skokloster-castle.glb proposed Semantic Scene Descriptor filename : data/scene_datasets/habitat-test-scenes/skokloster-castle.scn
E1001 20:37:12.295025 20600 SemanticScene.h:155] ::loadSemanticSceneDescriptor : File data/scene_datasets/habitat-test-scenes/skokloster-castle.scn does not exist.  Aborting load.
W1001 20:37:12.295037 20600 Simulator.cpp:275] ::setSceneInstanceAttributes : All attempts to load SSD with SceneAttributes-provided name data/scene_datasets/habitat-test-scenes/skokloster-castle.scn : exist : 0 : loaded as expected type : 0
I1001 20:37:12.295300 20600 Simulator.cpp:324] ::createSceneInstance : Using scene instance-specified Light key : -no_lights-
I1001 20:37:12.295312 20600 MetadataMediator.cpp:68] ::setSimulatorConfiguration : Set new simulator config for scene/stage : data/scene_datasets/habitat-test-scenes/skokloster-castle.glb and dataset : default which is currently active dataset.
I1001 20:37:12.295342 20600 Simulator.cpp:382] ::createSceneInstance : Start to load stage named : data/scene_datasets/habitat-test-scenes/skokloster-castle.glb with render asset : data/scene_datasets/habitat-test-scenes/skokloster-castle.glb and collision asset : data/scene_datasets/habitat-test-scenes/skokloster-castle.glb
I1001 20:37:12.295365 20600 ResourceManager.cpp:257] ::loadStage : Not loading semantic mesh
I1001 20:37:12.295372 20600 ResourceManager.cpp:285] ::loadStage : start load render asset data/scene_datasets/habitat-test-scenes/skokloster-castle.glb.
I1001 20:37:12.295378 20600 ResourceManager.cpp:699] ::loadStageInternal : Attempting to load stage data/scene_datasets/habitat-test-scenes/skokloster-castle.glb 
I1001 20:37:12.295399 20600 ResourceManager.cpp:1280] Importing Basis files as BC7 for skokloster-castle.glb
I1001 20:37:13.235186 20600 Simulator.cpp:400] ::createSceneInstance : Successfully loaded stage named : data/scene_datasets/habitat-test-scenes/skokloster-castle.glb
W1001 20:37:13.235219 20600 Simulator.cpp:435] 
---
Simulator::createSceneInstance : The active scene does not contain semantic annotations. 
---
I1001 20:37:13.235230 20600 MetadataMediator.cpp:262] ::getSceneAttributesByName : Query dataset : default for SceneAttributes named : data/scene_datasets/habitat-test-scenes/skokloster-castle.glb yields 1 candidates.  Using data/scene_datasets/habitat-test-scenes/skokloster-castle.glb.
I1001 20:37:13.235258 20600 SceneDatasetAttributes.cpp:45] ::addNewSceneInstanceToDataset : Dataset : 'default' : Stage Attributes 'data/scene_datasets/habitat-test-scenes/skokloster-castle.glb' specified in Scene Attributes exists in dataset library.
I1001 20:37:13.235263 20600 SceneDatasetAttributes.cpp:85] ::addNewSceneInstanceToDataset : Dataset : 'default' : Lighting Layout Attributes no_lights specified in Scene Attributes exists in dataset library.
I1001 20:37:13.235270 20600 MetadataMediator.cpp:262] ::getSceneAttributesByName : Query dataset : default for SceneAttributes named : data/scene_datasets/habitat-test-scenes/skokloster-castle.glb yields 1 candidates.  Using data/scene_datasets/habitat-test-scenes/skokloster-castle.glb.
I1001 20:37:13.235280 20600 SceneDatasetAttributes.cpp:45] ::addNewSceneInstanceToDataset : Dataset : 'default' : Stage Attributes 'data/scene_datasets/habitat-test-scenes/skokloster-castle.glb' specified in Scene Attributes exists in dataset library.
I1001 20:37:13.235285 20600 SceneDatasetAttributes.cpp:85] ::addNewSceneInstanceToDataset : Dataset : 'default' : Lighting Layout Attributes no_lights specified in Scene Attributes exists in dataset library.
I1001 20:37:13.235298 20600 Simulator.cpp:182] Simulator::reconfigure() : createSceneInstance success == true for active scene name : data/scene_datasets/habitat-test-scenes/skokloster-castle.glb with renderer.
I1001 20:37:13.235625 20600 simulator.py:221] Loaded navmesh data/scene_datasets/habitat-test-scenes/skokloster-castle.navmesh
2021-10-01 20:37:13,238 Initializing task Nav-v0
2021-10-01 20:37:13,510 agent number of parameters: 5821797
Traceback (most recent call last):
  File "habitat_baselines/run.py", line 81, in <module>
    main()
  File "habitat_baselines/run.py", line 40, in main
    run_exp(**vars(args))
  File "habitat_baselines/run.py", line 77, in run_exp
    execute_exp(config, run_type)
  File "habitat_baselines/run.py", line 60, in execute_exp
    trainer.train()
  File "/home/sdu310/anaconda3/envs/habitat/lib/python3.6/contextlib.py", line 52, in inner
    return func(*args, **kwds)
  File "/home/sdu310/habitat-lab/habitat_baselines/rl/ppo/ppo_trainer.py", line 712, in train
    self._init_train()
  File "/home/sdu310/habitat-lab/habitat_baselines/rl/ppo/ppo_trainer.py", line 315, in _init_train
    observations, device=self.device, cache=self._obs_batching_cache
  File "/home/sdu310/anaconda3/envs/habitat/lib/python3.6/site-packages/torch-1.9.1-py3.6-linux-x86_64.egg/torch/autograd/grad_mode.py", line 28, in decorate_context
    return func(*args, **kwargs)
  File "/home/sdu310/anaconda3/envs/habitat/lib/python3.6/contextlib.py", line 52, in inner
    return func(*args, **kwds)
  File "/home/sdu310/habitat-lab/habitat_baselines/utils/common.py", line 246, in batch_obs
    batch_t[sensor_name][i] = sensor
TypeError: can't assign a numpy.ndarray to a torch.ByteTensor
Exception ignored in: <bound method VectorEnv.__del__ of <habitat.core.vector_env.VectorEnv object at 0x7f071cb61d68>>
Traceback (most recent call last):
  File "/home/sdu310/habitat-lab/habitat/core/vector_env.py", line 592, in __del__
    self.close()
  File "/home/sdu310/habitat-lab/habitat/core/vector_env.py", line 463, in close
    write_fn((CLOSE_COMMAND, None))
  File "/home/sdu310/habitat-lab/habitat/core/vector_env.py", line 118, in __call__
    self.write_fn(data)
  File "/home/sdu310/habitat-lab/habitat/utils/pickle5_multiprocessing.py", line 63, in send
    self.send_bytes(buf.getvalue())
  File "/home/sdu310/anaconda3/envs/habitat/lib/python3.6/multiprocessing/connection.py", line 200, in send_bytes
    self._send_bytes(m[offset:offset + size])
  File "/home/sdu310/anaconda3/envs/habitat/lib/python3.6/multiprocessing/connection.py", line 404, in _send_bytes
    self._send(header + buf)
  File "/home/sdu310/anaconda3/envs/habitat/lib/python3.6/multiprocessing/connection.py", line 368, in _send
    n = write(self._handle, buf)
BrokenPipeError: [Errno 32] Broken pipe
```
  
  
I have tried uninstall and install again the numpy several times, but the problem doesnt change.  
And my python packages are listed :
```
(habitat) sdu310@sdu310-Precision-5820-Tower-X-Series:~/habitat-lab$ pip list
Package                 Version             Location
----------------------- ------------------- ------------------------
absl-py                 0.11.0
argon2-cffi             20.1.0
astor                   0.8.1
async-generator         1.10
attrs                   20.3.0
backcall                0.2.0
braceexpand             0.1.7
cached-property         1.5.2
cachetools              4.2.4
certifi                 2021.5.30
cffi                    1.14.4
charset-normalizer      2.0.6
click                   8.0.1
cloudpickle             1.6.0
colorama                0.4.4
cycler                  0.10.0
dataclasses             0.8
decorator               4.4.2
defusedxml              0.6.0
easydict                1.9
entrypoints             0.3
gast                    0.2.2
gitdb                   4.0.7
GitPython               3.1.18
google-auth             2.2.1
google-auth-oauthlib    0.4.6
google-pasta            0.2.0
grpcio                  1.33.2
gym                     0.18.3
h5py                    3.4.0
habitat                 0.2.1               /home/sdu310/habitat-lab
habitat-sim             0.2.1
idna                    3.2
ifcfg                   0.22
imageio                 2.9.0
imageio-ffmpeg          0.4.5
importlib-metadata      3.1.0
ipykernel               5.3.4
ipython                 7.16.1
ipython-genutils        0.2.0
ipywidgets              7.5.1
jedi                    0.17.2
Jinja2                  2.11.2
joblib                  0.17.0
jsonschema              3.2.0
jupyter-client          6.1.7
jupyter-core            4.7.0
jupyterlab-pygments     0.1.2
Keras-Applications      1.0.8
Keras-Preprocessing     1.1.2
kiwisolver              1.3.1
llvmlite                0.36.0
lmdb                    1.2.1
magnum                  0.0.0
Markdown                3.3.3
MarkupSafe              1.1.1
matplotlib              3.2.2
mistune                 0.8.4
mock                    4.0.3
moviepy                 2.0.0.dev2
msgpack                 1.0.2
nbclient                0.5.1
nbconvert               6.0.7
nbformat                5.0.8
nest-asyncio            1.4.3
notebook                6.1.5
numba                   0.53.1
numpy                   1.19.5
numpy-quaternion        2021.8.30.10.33.11
oauthlib                3.1.1
objectio                0.2.29
olefile                 0.46
open3d-python           0.7.0.0
opencv-python           4.4.0.46
opt-einsum              3.3.0
packaging               20.7
pandocfilters           1.4.3
parso                   0.7.1
pexpect                 4.8.0
pickle5                 0.0.11
pickleshare             0.7.5
Pillow                  8.2.0
pip                     21.2.2
proglog                 0.1.9
prometheus-client       0.9.0
prompt-toolkit          3.0.8
protobuf                3.14.0
ptyprocess              0.6.0
pyasn1                  0.4.8
pyasn1-modules          0.2.8
pycparser               2.20
pyglet                  1.5.15
Pygments                2.7.2
pyparsing               2.4.7
pyrsistent              0.17.3
python-dateutil         2.8.1
PyYAML                  5.4.1
pyzmq                   20.0.0
requests                2.26.0
requests-oauthlib       1.3.0
rsa                     4.7.2
scikit-learn            0.23.2
scipy                   1.5.4
Send2Trash              1.5.0
setuptools              52.0.0.post20210125
Shapely                 1.7.1
simplejson              3.17.5
six                     1.15.0
smmap                   3.0.5
tb-nightly              2.7.0a20210930
tensorboard             1.13.1
tensorboard-data-server 0.6.1
tensorboard-plugin-wit  1.8.0
tensorflow              1.13.1
tensorflow-estimator    1.13.0
termcolor               1.1.0
terminado               0.9.1
testpath                0.4.4
threadpoolctl           2.1.0
torch                   1.9.1
tornado                 6.1
tqdm                    4.54.0
traitlets               4.3.3
typer                   0.4.0
typing-extensions       3.10.0.0
urllib3                 1.26.7
wcwidth                 0.2.5
webdataset              0.1.40
webencodings            0.5.1
Werkzeug                1.0.1
wheel                   0.37.0
widgetsnbextension      3.5.1
wrapt                   1.12.1
yacs                    0.1.8
zipp                    3.4.0
```

My Python version is Python 3.6.13.

I have been confused for a whole day, I don't know how to solve it, please help me! Thanks!!!



Discussion:

Resolving patch:
diff --git a/habitat_baselines/utils/common.py b/habitat_baselines/utils/common.py
--- a/habitat_baselines/utils/common.py
+++ b/habitat_baselines/utils/common.py
@@ -180,9 +180,12 @@ def get(
             and device.type == "cuda"
             and cache.device.type == "cpu"
         ):
+            cache = cache.pin_memory()
+
+        if cache.device.type == "cpu":
             # Pytorch indexing is slow,
             # so convert to numpy
-            cache = cache.pin_memory().numpy()
+            cache = cache.numpy()
 
         self._pool[key] = cache
         return cache


Test patch:
diff --git a/test/test_baseline_trainers.py b/test/test_baseline_trainers.py
--- a/test/test_baseline_trainers.py
+++ b/test/test_baseline_trainers.py
@@ -24,7 +24,10 @@
     from habitat_baselines.common.baseline_registry import baseline_registry
     from habitat_baselines.config.default import get_config
     from habitat_baselines.run import execute_exp, run_exp
-    from habitat_baselines.utils.common import batch_obs
+    from habitat_baselines.utils.common import (
+        ObservationBatchingCache,
+        batch_obs,
+    )
 
     baseline_installed = True
 except ImportError:
@@ -328,3 +331,34 @@ def test_pausing():
     num_envs = 8
     __do_pause_test(num_envs, [])
     __do_pause_test(num_envs, list(range(num_envs)))
+
+
+@pytest.mark.skipif(
+    not baseline_installed, reason="baseline sub-module not installed"
+)
+@pytest.mark.parametrize(
+    "sensor_device,batched_device",
+    [("cpu", "cpu"), ("cpu", "cuda"), ("cuda", "cuda")],
+)
+def test_batch_obs(sensor_device, batched_device):
+    if (
+        "cuda" in (sensor_device, batched_device)
+        and not torch.cuda.is_available()
+    ):
+        pytest.skip("CUDA not avaliable")
+
+    sensor_device = torch.device(sensor_device)
+    batched_device = torch.device(batched_device)
+
+    numpy_if = lambda t: t.numpy() if sensor_device.type == "cpu" else t
+
+    cache = ObservationBatchingCache()
+    sensors = [
+        {
+            f"{s}": numpy_if(torch.randn(128, 128, device=sensor_device))
+            for s in range(4)
+        }
+        for _ in range(4)
+    ]
+
+    _ = batch_obs(sensors, device=batched_device, cache=cache)