Skip to content

D
detectron
Commit e5bb3a8f authored by Ilija Radosavovic's avatar Ilija Radosavovic Committed by Facebook Github Bot
Browse files
Options

Encapsulate detectron code in a package 

Reviewed By: rbgirshick

Differential Revision: D7516523

fbshipit-source-id: 7c67f82c7ce6f79a66b0bdea0770d2d00735e38f
parent 25d9e240
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 381 additions and 338 deletions
.gitignore
View file @ e5bb3a8f
......@@ -7,7 +7,7 @@ __pycache__/
*.so
# Distribution / packaging
lib/build/
build/
*.egg-info/
*.egg
......@@ -17,8 +17,8 @@ lib/build/
*.swp
# Dataset symlinks
lib/datasets/data/*
!lib/datasets/data/README.md
detectron/datasets/data/*
!detectron/datasets/data/README.md
# Generated C files
lib/utils/cython_*.c
detectron/utils/cython_*.c
lib/CMakeLists.txt → CMakeLists.txt
View file @ e5bb3a8f
......@@ -26,8 +26,8 @@ include(cmake/Summary.cmake)
detectron_print_config_summary()
# Collect custom ops sources.
file(GLOB CUSTOM_OPS_CPU_SRCS ${CMAKE_CURRENT_SOURCE_DIR}/ops/*.cc)
file(GLOB CUSTOM_OPS_GPU_SRCS ${CMAKE_CURRENT_SOURCE_DIR}/ops/*.cu)
file(GLOB CUSTOM_OPS_CPU_SRCS ${CMAKE_CURRENT_SOURCE_DIR}/detectron/ops/*.cc)
file(GLOB CUSTOM_OPS_GPU_SRCS ${CMAKE_CURRENT_SOURCE_DIR}/detectron/ops/*.cu)
# Install custom CPU ops lib.
add_library(
......
FAQ.md
View file @ e5bb3a8f
......@@ -23,7 +23,7 @@ Also potentially relevant: inference with Mask R-CNN on high-resolution images m
#### Q: How do I implement a custom Caffe2 CPU or GPU operator for use in Detectron?
**A:** Detectron uses a number of specialized Caffe2 operators that are distributed via the [Caffe2 Detectron module](https://github.com/caffe2/caffe2/tree/master/modules/detectron) as part of the core Caffe2 GitHub repository. If you'd like to implement a custom Caffe2 operator for your project, we have written a toy example illustrating how to add an operator under the Detectron source tree; please see [`lib/ops/zero_even_op.*`](lib/ops/) and [`tests/test_zero_even_op.py`](tests/test_zero_even_op.py). For more background on writing Caffe2 operators please consult the [Caffe2 documentation](https://caffe2.ai/docs/custom-operators.html).
**A:** Detectron uses a number of specialized Caffe2 operators that are distributed via the [Caffe2 Detectron module](https://github.com/caffe2/caffe2/tree/master/modules/detectron) as part of the core Caffe2 GitHub repository. If you'd like to implement a custom Caffe2 operator for your project, we have written a toy example illustrating how to add an operator under the Detectron source tree; please see [`detectron/ops/zero_even_op.*`](detectron/ops/) and [`detectron/tests/test_zero_even_op.py`](detectron/tests/test_zero_even_op.py). For more background on writing Caffe2 operators please consult the [Caffe2 documentation](https://caffe2.ai/docs/custom-operators.html).
#### Q: How do I use Detectron to train a model on a custom dataset?
......
INSTALL.md
View file @ e5bb3a8f
......@@ -66,13 +66,13 @@ git clone https://github.com/facebookresearch/detectron $DETECTRON
Set up Python modules:
```
cd $DETECTRON/lib && make
cd $DETECTRON && make
```
Check that Detectron tests pass (e.g. for [`SpatialNarrowAsOp test`](tests/test_spatial_narrow_as_op.py)):
```
python2 $DETECTRON/tests/test_spatial_narrow_as_op.py
python2 $DETECTRON/detectron/tests/test_spatial_narrow_as_op.py
```
## That's All You Need for Inference
......@@ -81,7 +81,7 @@ At this point, you can run inference using pretrained Detectron models. Take a l
## Datasets
Detectron finds datasets via symlinks from `lib/datasets/data` to the actual locations where the dataset images and annotations are stored. For instructions on how to create symlinks for COCO and other datasets, please see [`lib/datasets/data/README.md`](lib/datasets/data/README.md).
Detectron finds datasets via symlinks from `detectron/datasets/data` to the actual locations where the dataset images and annotations are stored. For instructions on how to create symlinks for COCO and other datasets, please see [`detectron/datasets/data/README.md`](detectron/datasets/data/README.md).
After symlinks have been created, that's all you need to start training models.
......@@ -90,18 +90,18 @@ After symlinks have been created, that's all you need to start training models.
Please read the custom operators section of the [`FAQ`](FAQ.md) first.
For convenience, we provide CMake support for building custom operators. All custom operators are built into a single library that can be loaded dynamically from Python.
Place your custom operator implementation under [`lib/ops/`](lib/ops/) and see [`tests/test_zero_even_op.py`](tests/test_zero_even_op.py) for an example of how to load custom operators from Python.
Place your custom operator implementation under [`detectron/ops/`](detectron/ops/) and see [`detectron/tests/test_zero_even_op.py`](detectron/tests/test_zero_even_op.py) for an example of how to load custom operators from Python.
Build the custom operators library:
```
cd $DETECTRON/lib && make ops
cd $DETECTRON && make ops
```
Check that the custom operator tests pass:
```
python2 $DETECTRON/tests/test_zero_even_op.py
python2 $DETECTRON/detectron/tests/test_zero_even_op.py
```
## Docker Image
......@@ -118,7 +118,7 @@ docker build -t detectron:c2-cuda9-cudnn7 .
Run the image (e.g. for [`BatchPermutationOp test`](tests/test_batch_permutation_op.py)):
```
nvidia-docker run --rm -it detectron:c2-cuda9-cudnn7 python2 tests/test_batch_permutation_op.py
nvidia-docker run --rm -it detectron:c2-cuda9-cudnn7 python2 detectron/tests/test_batch_permutation_op.py
```
## Troubleshooting
......@@ -149,7 +149,7 @@ cmake .. \
Similarly, when building custom Detectron operators you can use:
```
cd $DETECTRON/lib
cd $DETECTRON
mkdir -p build && cd build
cmake .. \
-DCUDA_TOOLKIT_ROOT_DIR=/path/to/cuda/toolkit/dir \
......
lib/cmake/legacy/legacymake.cmake → cmake/legacy/legacymake.cmake
View file @ e5bb3a8f
......@@ -19,8 +19,8 @@ include(cmake/Summary.cmake)
detectron_print_config_summary()
# Collect custom ops sources.
file(GLOB CUSTOM_OPS_CPU_SRCS ${CMAKE_CURRENT_SOURCE_DIR}/ops/*.cc)
file(GLOB CUSTOM_OPS_GPU_SRCS ${CMAKE_CURRENT_SOURCE_DIR}/ops/*.cu)
file(GLOB CUSTOM_OPS_CPU_SRCS ${CMAKE_CURRENT_SOURCE_DIR}/detectron/ops/*.cc)
file(GLOB CUSTOM_OPS_GPU_SRCS ${CMAKE_CURRENT_SOURCE_DIR}/detectron/ops/*.cu)
# Install custom CPU ops lib.
add_library(
......
lib/core/config.py → detectron/core/config.py
View file @ e5bb3a8f
......@@ -43,8 +43,8 @@ from __future__ import print_function
from __future__ import unicode_literals
from ast import literal_eval
from future.utils import iteritems
from past.builtins import basestring
from utils.collections import AttrDict
import copy
import logging
import numpy as np
......@@ -52,13 +52,14 @@ import os
import os.path as osp
import yaml
from utils.io import cache_url
from detectron.utils.collections import AttrDict
from detectron.utils.io import cache_url
logger = logging.getLogger(__name__)
__C = AttrDict()
# Consumers can get config by:
# from core.config import cfg
# from detectron.core.config import cfg
cfg = __C
......@@ -1000,6 +1001,7 @@ _DEPCRECATED_KEYS = set(
}
)
# ---------------------------------------------------------------------------- #
# Renamed options
# If you rename a config option, record the mapping from the old name to the new
......@@ -1044,6 +1046,17 @@ _RENAMED_KEYS = {
}
# ---------------------------------------------------------------------------- #
# Renamed modules
# If a module containing a data structure used in the config (e.g. AttrDict)
# is renamed/moved and you don't want to break loading of existing yaml configs
# (e.g. from weights files) you can specify the renamed module below.
# ---------------------------------------------------------------------------- #
_RENAMED_MODULES = {
'utils.collections': 'detectron.utils.collections',
}
def assert_and_infer_cfg(cache_urls=True, make_immutable=True):
"""Call this function in your script after you have finished setting all cfg
values that are necessary (e.g., merging a config from a file, merging
......@@ -1090,10 +1103,24 @@ def get_output_dir(datasets, training=True):
return outdir
def load_cfg(cfg_to_load):
"""Wrapper around yaml.load used for maintaining backward compatibility"""
assert isinstance(cfg_to_load, (file, basestring)), \
'Expected {} or {} got {}'.format(file, basestring, type(cfg_to_load))
if isinstance(cfg_to_load, file):
cfg_to_load = ''.join(cfg_to_load.readlines())
if isinstance(cfg_to_load, basestring):
for old_module, new_module in iteritems(_RENAMED_MODULES):
# yaml object encoding: !!python/object/new:<module>.<object>
old_module, new_module = 'new:' + old_module, 'new:' + new_module
cfg_to_load = cfg_to_load.replace(old_module, new_module)
return yaml.load(cfg_to_load)
def merge_cfg_from_file(cfg_filename):
"""Load a yaml config file and merge it into the global config."""
with open(cfg_filename, 'r') as f:
yaml_cfg = AttrDict(yaml.load(f))
yaml_cfg = AttrDict(load_cfg(f))
_merge_a_into_b(yaml_cfg, __C)
......@@ -1130,8 +1157,10 @@ def _merge_a_into_b(a, b, stack=None):
"""Merge config dictionary a into config dictionary b, clobbering the
options in b whenever they are also specified in a.
"""
assert isinstance(a, AttrDict), 'Argument `a` must be an AttrDict'
assert isinstance(b, AttrDict), 'Argument `b` must be an AttrDict'
assert isinstance(a, AttrDict), \
'`a` (cur type {}) must be an instance of {}'.format(type(a), AttrDict)
assert isinstance(b, AttrDict), \
'`b` (cur type {}) must be an instance of {}'.format(type(b), AttrDict)
for k, v_ in a.items():
full_key = '.'.join(stack) + '.' + k if stack is not None else k
......
lib/core/rpn_generator.py → detectron/core/rpn_generator.py
View file @ e5bb3a8f
......@@ -38,17 +38,17 @@ import yaml
from caffe2.python import core
from caffe2.python import workspace
from core.config import cfg
from datasets import task_evaluation
from datasets.json_dataset import JsonDataset
from modeling import model_builder
from utils.io import save_object
from utils.timer import Timer
import utils.blob as blob_utils
import utils.c2 as c2_utils
import utils.env as envu
import utils.net as nu
import utils.subprocess as subprocess_utils
from detectron.core.config import cfg
from detectron.datasets import task_evaluation
from detectron.datasets.json_dataset import JsonDataset
from detectron.modeling import model_builder
from detectron.utils.io import save_object
from detectron.utils.timer import Timer
import detectron.utils.blob as blob_utils
import detectron.utils.c2 as c2_utils
import detectron.utils.env as envu
import detectron.utils.net as nu
import detectron.utils.subprocess as subprocess_utils
logger = logging.getLogger(__name__)
......
lib/core/test.py → detectron/core/test.py
View file @ e5bb3a8f
......@@ -37,14 +37,14 @@ from caffe2.python import core
from caffe2.python import workspace
import pycocotools.mask as mask_util
from core.config import cfg
from utils.timer import Timer
import core.test_retinanet as test_retinanet
import modeling.FPN as fpn
import utils.blob as blob_utils
import utils.boxes as box_utils
import utils.image as image_utils
import utils.keypoints as keypoint_utils
from detectron.core.config import cfg
from detectron.utils.timer import Timer
import detectron.core.test_retinanet as test_retinanet
import detectron.modeling.FPN as fpn
import detectron.utils.blob as blob_utils
import detectron.utils.boxes as box_utils
import detectron.utils.image as image_utils
import detectron.utils.keypoints as keypoint_utils
logger = logging.getLogger(__name__)
......
lib/core/test_engine.py → detectron/core/test_engine.py
View file @ e5bb3a8f
......@@ -30,21 +30,21 @@ import yaml
from caffe2.python import workspace
from core.config import cfg
from core.config import get_output_dir
from core.rpn_generator import generate_rpn_on_dataset
from core.rpn_generator import generate_rpn_on_range
from core.test import im_detect_all
from datasets import task_evaluation
from datasets.json_dataset import JsonDataset
from modeling import model_builder
from utils.io import save_object
from utils.timer import Timer
import utils.c2 as c2_utils
import utils.env as envu
import utils.net as net_utils
import utils.subprocess as subprocess_utils
import utils.vis as vis_utils
from detectron.core.config import cfg
from detectron.core.config import get_output_dir
from detectron.core.rpn_generator import generate_rpn_on_dataset
from detectron.core.rpn_generator import generate_rpn_on_range
from detectron.core.test import im_detect_all
from detectron.datasets import task_evaluation
from detectron.datasets.json_dataset import JsonDataset
from detectron.modeling import model_builder
from detectron.utils.io import save_object
from detectron.utils.timer import Timer
import detectron.utils.c2 as c2_utils
import detectron.utils.env as envu
import detectron.utils.net as net_utils
import detectron.utils.subprocess as subprocess_utils
import detectron.utils.vis as vis_utils
logger = logging.getLogger(__name__)
......
lib/core/test_retinanet.py → detectron/core/test_retinanet.py
View file @ e5bb3a8f
......@@ -26,12 +26,11 @@ from collections import defaultdict
from caffe2.python import core, workspace
from core.config import cfg
from modeling.generate_anchors import generate_anchors
from utils.timer import Timer
import utils.blob as blob_utils
import utils.boxes as box_utils
from detectron.core.config import cfg
from detectron.modeling.generate_anchors import generate_anchors
from detectron.utils.timer import Timer
import detectron.utils.blob as blob_utils
import detectron.utils.boxes as box_utils
logger = logging.getLogger(__name__)
......
lib/datasets/cityscapes/tools/convert_cityscapes_to_coco.py → detectron/datasets/cityscapes/tools/convert_cityscapes_to_coco.py
View file @ e5bb3a8f
......@@ -12,8 +12,8 @@ import sys
import cityscapesscripts.evaluation.instances2dict_with_polygons as cs
import utils.segms as segms_util
import utils.boxes as bboxs_util
import detectron.utils.segms as segms_util
import detectron.utils.boxes as bboxs_util
def parse_args():
......
lib/datasets/cityscapes/tools/convert_coco_model_to_cityscapes.py → detectron/datasets/cityscapes/tools/convert_coco_model_to_cityscapes.py
View file @ e5bb3a8f
......@@ -8,13 +8,13 @@ from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals
import cPickle as pickle
import argparse
import cPickle as pickle
import numpy as np
import os
import sys
import numpy as np
import datasets.cityscapes.coco_to_cityscapes_id as cs
import detectron.datasets.cityscapes.coco_to_cityscapes_id as cs
NUM_CS_CLS = 9
NUM_COCO_CLS = 81
......
lib/datasets/cityscapes_json_dataset_evaluator.py → detectron/datasets/cityscapes_json_dataset_evaluator.py
View file @ e5bb3a8f
......@@ -27,9 +27,9 @@ import uuid
import pycocotools.mask as mask_util
from core.config import cfg
from datasets.dataset_catalog import DATASETS
from datasets.dataset_catalog import RAW_DIR
from detectron.core.config import cfg
from detectron.datasets.dataset_catalog import DATASETS
from detectron.datasets.dataset_catalog import RAW_DIR
logger = logging.getLogger(__name__)
......
lib/datasets/data/README.md → detectron/datasets/data/README.md
View file @ e5bb3a8f
......@@ -7,7 +7,7 @@ This directory contains symlinks to data locations.
Symlink the COCO dataset:
```
ln -s /path/to/coco $DETECTRON/lib/datasets/data/coco
ln -s /path/to/coco $DETECTRON/detectron/datasets/data/coco
```
We assume that your local COCO dataset copy at `/path/to/coco` has the following directory structure:
......@@ -28,10 +28,10 @@ coco
If that is not the case, you may need to do something similar to:
```
mkdir -p $DETECTRON/lib/datasets/data/coco
ln -s /path/to/coco_train2014 $DETECTRON/lib/datasets/data/coco/
ln -s /path/to/coco_val2014 $DETECTRON/lib/datasets/data/coco/
ln -s /path/to/json/annotations $DETECTRON/lib/datasets/data/coco/annotations
mkdir -p $DETECTRON/detectron/datasets/data/coco
ln -s /path/to/coco_train2014 $DETECTRON/detectron/datasets/data/coco/
ln -s /path/to/coco_val2014 $DETECTRON/detectron/datasets/data/coco/
ln -s /path/to/json/annotations $DETECTRON/detectron/datasets/data/coco/annotations
```
### COCO Minival Annotations
......@@ -42,7 +42,7 @@ Similarly, the union of `valminusminival` and the 2014 `train` is exactly equiva
## Creating Symlinks for PASCAL VOC
We assume that your symlinked `lib/datasets/data/VOC<year>` directory has the following structure:
We assume that your symlinked `detectron/datasets/data/VOC<year>` directory has the following structure:
```
VOC<year>
......@@ -59,10 +59,10 @@ VOC<year>
Create symlinks for `VOC<year>`:
```
mkdir -p $DETECTRON/lib/datasets/data/VOC<year>
ln -s /path/to/VOC<year>/JPEGImages $DETECTRON/lib/datasets/data/VOC<year>/JPEGImages
ln -s /path/to/VOC<year>/json/annotations $DETECTRON/lib/datasets/data/VOC<year>/annotations
ln -s /path/to/VOC<year>/devkit $DETECTRON/lib/datasets/data/VOC<year>/VOCdevkit<year>
mkdir -p $DETECTRON/detectron/datasets/data/VOC<year>
ln -s /path/to/VOC<year>/JPEGImages $DETECTRON/detectron/datasets/data/VOC<year>/JPEGImages
ln -s /path/to/VOC<year>/json/annotations $DETECTRON/detectron/datasets/data/VOC<year>/annotations
ln -s /path/to/VOC<year>/devkit $DETECTRON/detectron/datasets/data/VOC<year>/VOCdevkit<year>
```
### PASCAL VOC Annotations in COCO Format
......@@ -71,7 +71,7 @@ We expect PASCAL VOC annotations converted to COCO json format, which are availa
## Creating Symlinks for Cityscapes:
We assume that your symlinked `lib/datasets/data/cityscapes` directory has the following structure:
We assume that your symlinked `detectron/datasets/data/cityscapes` directory has the following structure:
```
cityscapes
......@@ -91,10 +91,10 @@ cityscapes
Create symlinks for `cityscapes`:
```
mkdir -p $DETECTRON/lib/datasets/data/cityscapes
ln -s /path/to/cityscapes/images $DETECTRON/lib/datasets/data/cityscapes/images
ln -s /path/to/cityscapes/json/annotations $DETECTRON/lib/datasets/data/cityscapes/annotations
ln -s /path/to/cityscapes/root $DETECTRON/lib/datasets/data/cityscapes/raw
mkdir -p $DETECTRON/detectron/datasets/data/cityscapes
ln -s /path/to/cityscapes/images $DETECTRON/detectron/datasets/data/cityscapes/images
ln -s /path/to/cityscapes/json/annotations $DETECTRON/detectron/datasets/data/cityscapes/annotations
ln -s /path/to/cityscapes/root $DETECTRON/detectron/datasets/data/cityscapes/raw
```
### Cityscapes Annotations in COCO Format
......
lib/datasets/dummy_datasets.py → detectron/datasets/dummy_datasets.py
View file @ e5bb3a8f
......@@ -22,7 +22,7 @@ from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals
from utils.collections import AttrDict
from detectron.utils.collections import AttrDict
def get_coco_dataset():
......
lib/datasets/json_dataset.py → detectron/datasets/json_dataset.py
View file @ e5bb3a8f
......@@ -33,19 +33,19 @@ import os
import scipy.sparse
# Must happen before importing COCO API (which imports matplotlib)
import utils.env as envu
import detectron.utils.env as envu
envu.set_up_matplotlib()
# COCO API
from pycocotools import mask as COCOmask
from pycocotools.coco import COCO
from core.config import cfg
from datasets.dataset_catalog import ANN_FN
from datasets.dataset_catalog import DATASETS
from datasets.dataset_catalog import IM_DIR
from datasets.dataset_catalog import IM_PREFIX
from utils.timer import Timer
import utils.boxes as box_utils
from detectron.core.config import cfg
from detectron.datasets.dataset_catalog import ANN_FN
from detectron.datasets.dataset_catalog import DATASETS
from detectron.datasets.dataset_catalog import IM_DIR
from detectron.datasets.dataset_catalog import IM_PREFIX
from detectron.utils.timer import Timer
import detectron.utils.boxes as box_utils
logger = logging.getLogger(__name__)
......
lib/datasets/json_dataset_evaluator.py → detectron/datasets/json_dataset_evaluator.py
View file @ e5bb3a8f
......@@ -28,9 +28,9 @@ import uuid
from pycocotools.cocoeval import COCOeval
from core.config import cfg
from utils.io import save_object
import utils.boxes as box_utils
from detectron.core.config import cfg
from detectron.utils.io import save_object
import detectron.utils.boxes as box_utils
logger = logging.getLogger(__name__)
......
lib/datasets/roidb.py → detectron/datasets/roidb.py
View file @ e5bb3a8f
......@@ -24,11 +24,11 @@ from past.builtins import basestring
import logging
import numpy as np
from core.config import cfg
from datasets.json_dataset import JsonDataset
import utils.boxes as box_utils
import utils.keypoints as keypoint_utils
import utils.segms as segm_utils
from detectron.core.config import cfg
from detectron.datasets.json_dataset import JsonDataset
import detectron.utils.boxes as box_utils
import detectron.utils.keypoints as keypoint_utils
import detectron.utils.segms as segm_utils
logger = logging.getLogger(__name__)
......
lib/datasets/task_evaluation.py → detectron/datasets/task_evaluation.py
View file @ e5bb3a8f
......@@ -40,11 +40,12 @@ import logging
import os
import pprint
from core.config import cfg
from utils.logging import send_email
import datasets.cityscapes_json_dataset_evaluator as cs_json_dataset_evaluator
import datasets.json_dataset_evaluator as json_dataset_evaluator
import datasets.voc_dataset_evaluator as voc_dataset_evaluator
from detectron.core.config import cfg
from detectron.utils.logging import send_email
import detectron.datasets.cityscapes_json_dataset_evaluator \
as cs_json_dataset_evaluator
import detectron.datasets.json_dataset_evaluator as json_dataset_evaluator
import detectron.datasets.voc_dataset_evaluator as voc_dataset_evaluator
logger = logging.getLogger(__name__)
......
lib/datasets/voc_dataset_evaluator.py → detectron/datasets/voc_dataset_evaluator.py
View file @ e5bb3a8f
......@@ -26,11 +26,11 @@ import os
import shutil
import uuid
from core.config import cfg
from datasets.dataset_catalog import DATASETS
from datasets.dataset_catalog import DEVKIT_DIR
from datasets.voc_eval import voc_eval
from utils.io import save_object
from detectron.core.config import cfg
from detectron.datasets.dataset_catalog import DATASETS
from detectron.datasets.dataset_catalog import DEVKIT_DIR
from detectron.datasets.voc_eval import voc_eval
from detectron.utils.io import save_object
logger = logging.getLogger(__name__)
......
lib/modeling/FPN.py → detectron/modeling/FPN.py
View file @ e5bb3a8f
......@@ -23,14 +23,14 @@ from __future__ import unicode_literals
import collections
import numpy as np
from core.config import cfg
from modeling.generate_anchors import generate_anchors
from utils.c2 import const_fill
from utils.c2 import gauss_fill
from utils.net import get_group_gn
import modeling.ResNet as ResNet
import utils.blob as blob_utils
import utils.boxes as box_utils
from detectron.core.config import cfg
from detectron.modeling.generate_anchors import generate_anchors
from detectron.utils.c2 import const_fill
from detectron.utils.c2 import gauss_fill
from detectron.utils.net import get_group_gn
import detectron.modeling.ResNet as ResNet
import detectron.utils.blob as blob_utils
import detectron.utils.boxes as box_utils
# Lowest and highest pyramid levels in the backbone network. For FPN, we assume
# that all networks have 5 spatial reductions, each by a factor of 2. Level 1
......
lib/modeling/ResNet.py → detectron/modeling/ResNet.py
View file @ e5bb3a8f
......@@ -23,8 +23,9 @@ from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals
from core.config import cfg
from utils.net import get_group_gn
from detectron.core.config import cfg
from detectron.utils.net import get_group_gn
# ---------------------------------------------------------------------------- #
# Bits for specific architectures (ResNet50, ResNet101, ...)
......
lib/modeling/VGG16.py → detectron/modeling/VGG16.py
View file @ e5bb3a8f
......@@ -20,7 +20,7 @@ from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals
from core.config import cfg
from detectron.core.config import cfg
def add_VGG16_conv5_body(model):
......
lib/modeling/VGG_CNN_M_1024.py → detectron/modeling/VGG_CNN_M_1024.py
View file @ e5bb3a8f
......@@ -20,7 +20,7 @@ from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals
from core.config import cfg
from detectron.core.config import cfg
def add_VGG_CNN_M_1024_conv5_body(model):
......
lib/modeling/detector.py → detectron/modeling/detector.py
View file @ e5bb3a8f
......@@ -29,13 +29,13 @@ from caffe2.python import workspace
from caffe2.python.modeling import initializers
from caffe2.python.modeling.parameter_info import ParameterTags
from core.config import cfg
from ops.collect_and_distribute_fpn_rpn_proposals \
from detectron.core.config import cfg
from detectron.ops.collect_and_distribute_fpn_rpn_proposals \
import CollectAndDistributeFpnRpnProposalsOp
from ops.generate_proposal_labels import GenerateProposalLabelsOp
from ops.generate_proposals import GenerateProposalsOp
import roi_data.fast_rcnn
import utils.c2 as c2_utils
from detectron.ops.generate_proposal_labels import GenerateProposalLabelsOp
from detectron.ops.generate_proposals import GenerateProposalsOp
import detectron.roi_data.fast_rcnn as fast_rcnn_roi_data
import detectron.utils.c2 as c2_utils
logger = logging.getLogger(__name__)
......@@ -159,7 +159,7 @@ class DetectionModelHelper(cnn.CNNModelHelper):
# The list of blobs is not known before run-time because it depends on
# the specific model being trained. Query the data loader to get the
# list of output blob names.
blobs_out = roi_data.fast_rcnn.get_fast_rcnn_blob_names(
blobs_out = fast_rcnn_roi_data.get_fast_rcnn_blob_names(
is_training=self.train
)
blobs_out = [core.ScopedBlobReference(b) for b in blobs_out]
......@@ -214,7 +214,7 @@ class DetectionModelHelper(cnn.CNNModelHelper):
)
# Prepare output blobs
blobs_out = roi_data.fast_rcnn.get_fast_rcnn_blob_names(
blobs_out = fast_rcnn_roi_data.get_fast_rcnn_blob_names(
is_training=self.train
)
blobs_out = [core.ScopedBlobReference(b) for b in blobs_out]
......
lib/modeling/fast_rcnn_heads.py → detectron/modeling/fast_rcnn_heads.py
View file @ e5bb3a8f
......@@ -32,11 +32,11 @@ from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals
from core.config import cfg
from utils.c2 import const_fill
from utils.c2 import gauss_fill
from utils.net import get_group_gn
import utils.blob as blob_utils
from detectron.core.config import cfg
from detectron.utils.c2 import const_fill
from detectron.utils.c2 import gauss_fill
from detectron.utils.net import get_group_gn
import detectron.utils.blob as blob_utils
# ---------------------------------------------------------------------------- #
......
lib/modeling/keypoint_rcnn_heads.py → detectron/modeling/keypoint_rcnn_heads.py
View file @ e5bb3a8f
......@@ -32,11 +32,11 @@ from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals
from core.config import cfg
from utils.c2 import const_fill
from utils.c2 import gauss_fill
import modeling.ResNet as ResNet
import utils.blob as blob_utils
from detectron.core.config import cfg
from detectron.utils.c2 import const_fill
from detectron.utils.c2 import gauss_fill
import detectron.modeling.ResNet as ResNet
import detectron.utils.blob as blob_utils
# ---------------------------------------------------------------------------- #
......
lib/modeling/mask_rcnn_heads.py → detectron/modeling/mask_rcnn_heads.py
View file @ e5bb3a8f
......@@ -32,12 +32,12 @@ from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals
from core.config import cfg
from utils.c2 import const_fill
from utils.c2 import gauss_fill
from utils.net import get_group_gn
import modeling.ResNet as ResNet
import utils.blob as blob_utils
from detectron.core.config import cfg
from detectron.utils.c2 import const_fill
from detectron.utils.c2 import gauss_fill
from detectron.utils.net import get_group_gn
import detectron.modeling.ResNet as ResNet
import detectron.utils.blob as blob_utils
# ---------------------------------------------------------------------------- #
......
lib/modeling/model_builder.py → detectron/modeling/model_builder.py
View file @ e5bb3a8f
......@@ -42,19 +42,19 @@ import logging
from caffe2.python import core
from caffe2.python import workspace
from core.config import cfg
from modeling.detector import DetectionModelHelper
from roi_data.loader import RoIDataLoader
import modeling.fast_rcnn_heads as fast_rcnn_heads
import modeling.keypoint_rcnn_heads as keypoint_rcnn_heads
import modeling.mask_rcnn_heads as mask_rcnn_heads
import modeling.name_compat
import modeling.optimizer as optim
import modeling.retinanet_heads as retinanet_heads
import modeling.rfcn_heads as rfcn_heads
import modeling.rpn_heads as rpn_heads
import roi_data.minibatch
import utils.c2 as c2_utils
from detectron.core.config import cfg
from detectron.modeling.detector import DetectionModelHelper
from detectron.roi_data.loader import RoIDataLoader
import detectron.modeling.fast_rcnn_heads as fast_rcnn_heads
import detectron.modeling.keypoint_rcnn_heads as keypoint_rcnn_heads
import detectron.modeling.mask_rcnn_heads as mask_rcnn_heads
import detectron.modeling.name_compat as name_compat
import detectron.modeling.optimizer as optim
import detectron.modeling.retinanet_heads as retinanet_heads
import detectron.modeling.rfcn_heads as rfcn_heads
import detectron.modeling.rpn_heads as rpn_heads
import detectron.roi_data.minibatch as roi_data_minibatch
import detectron.utils.c2 as c2_utils
logger = logging.getLogger(__name__)
......@@ -131,7 +131,7 @@ def get_func(func_name):
"""
if func_name == '':
return None
new_func_name = modeling.name_compat.get_new_name(func_name)
new_func_name = name_compat.get_new_name(func_name)
if new_func_name != func_name:
logger.warn(
'Remapping old function name: {} -> {}'.
......@@ -144,7 +144,7 @@ def get_func(func_name):
if len(parts) == 1:
return globals()[parts[0]]
# Otherwise, assume we're referencing a module under modeling
module_name = 'modeling.' + '.'.join(parts[:-1])
module_name = 'detectron.modeling.' + '.'.join(parts[:-1])
module = importlib.import_module(module_name)
return getattr(module, parts[-1])
except Exception:
......@@ -388,9 +388,7 @@ def add_training_inputs(model, roidb=None):
blobs_queue_capacity=cfg.DATA_LOADER.BLOBS_QUEUE_CAPACITY
)
orig_num_op = len(model.net._net.op)
blob_names = roi_data.minibatch.get_minibatch_blob_names(
is_training=True
)
blob_names = roi_data_minibatch.get_minibatch_blob_names(is_training=True)
for gpu_id in range(cfg.NUM_GPUS):
with c2_utils.NamedCudaScope(gpu_id):
for blob_name in blob_names:
......@@ -433,9 +431,9 @@ def add_inference_inputs(model):
#
# ---------------------------------------------------------------------------- #
import modeling.ResNet as ResNet
import modeling.VGG16 as VGG16
import modeling.VGG_CNN_M_1024 as VGG_CNN_M_1024
import detectron.modeling.ResNet as ResNet
import detectron.modeling.VGG16 as VGG16
import detectron.modeling.VGG_CNN_M_1024 as VGG_CNN_M_1024
def fast_rcnn(model):
......
lib/modeling/optimizer.py → detectron/modeling/optimizer.py
View file @ e5bb3a8f
......@@ -24,8 +24,8 @@ import logging
from caffe2.python import muji
from core.config import cfg
import utils.c2 as c2_utils
from detectron.core.config import cfg
import detectron.utils.c2 as c2_utils
logger = logging.getLogger(__name__)
......
lib/modeling/retinanet_heads.py → detectron/modeling/retinanet_heads.py
View file @ e5bb3a8f
......@@ -22,8 +22,8 @@ from __future__ import unicode_literals
import numpy as np
from core.config import cfg
import utils.blob as blob_utils
from detectron.core.config import cfg
import detectron.utils.blob as blob_utils
def get_retinanet_bias_init(model):
......
lib/modeling/rfcn_heads.py → detectron/modeling/rfcn_heads.py
View file @ e5bb3a8f
......@@ -18,9 +18,9 @@ from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals
from core.config import cfg
from utils.c2 import const_fill
from utils.c2 import gauss_fill
from detectron.core.config import cfg
from detectron.utils.c2 import const_fill
from detectron.utils.c2 import gauss_fill
# ---------------------------------------------------------------------------- #
......
lib/modeling/rpn_heads.py → detectron/modeling/rpn_heads.py
View file @ e5bb3a8f
......@@ -18,12 +18,12 @@ from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals
from core.config import cfg
from modeling.generate_anchors import generate_anchors
from utils.c2 import const_fill
from utils.c2 import gauss_fill
import modeling.FPN as FPN
import utils.blob as blob_utils
from detectron.core.config import cfg
from detectron.modeling.generate_anchors import generate_anchors
from detectron.utils.c2 import const_fill
from detectron.utils.c2 import gauss_fill
import detectron.modeling.FPN as FPN
import detectron.utils.blob as blob_utils
# ---------------------------------------------------------------------------- #
......
lib/ops/collect_and_distribute_fpn_rpn_proposals.py → detectron/ops/collect_and_distribute_fpn_rpn_proposals.py
View file @ e5bb3a8f
......@@ -20,11 +20,11 @@ from __future__ import unicode_literals
import numpy as np
from core.config import cfg
from datasets import json_dataset
import modeling.FPN as fpn
import roi_data.fast_rcnn
import utils.blob as blob_utils
from detectron.core.config import cfg
from detectron.datasets import json_dataset
import detectron.modeling.FPN as fpn
import detectron.roi_data.fast_rcnn as fast_rcnn_roi_data
import detectron.utils.blob as blob_utils
class CollectAndDistributeFpnRpnProposalsOp(object):
......@@ -55,9 +55,9 @@ class CollectAndDistributeFpnRpnProposalsOp(object):
json_dataset.add_proposals(roidb, rois, im_scales, crowd_thresh=0)
# Compute training labels for the RPN proposals; also handles
# distributing the proposals over FPN levels
output_blob_names = roi_data.fast_rcnn.get_fast_rcnn_blob_names()
output_blob_names = fast_rcnn_roi_data.get_fast_rcnn_blob_names()
blobs = {k: [] for k in output_blob_names}
roi_data.fast_rcnn.add_fast_rcnn_blobs(blobs, im_scales, roidb)
fast_rcnn_roi_data.add_fast_rcnn_blobs(blobs, im_scales, roidb)
for i, k in enumerate(output_blob_names):
blob_utils.py_op_copy_blob(blobs[k], outputs[i])
else:
......@@ -88,7 +88,7 @@ def collect(inputs, is_training):
def distribute(rois, label_blobs, outputs, train):
"""To understand the output blob order see return value of
roi_data.fast_rcnn.get_fast_rcnn_blob_names(is_training=False)
detectron.roi_data.fast_rcnn.get_fast_rcnn_blob_names(is_training=False)
"""
lvl_min = cfg.FPN.ROI_MIN_LEVEL
lvl_max = cfg.FPN.ROI_MAX_LEVEL
......
lib/ops/generate_proposal_labels.py → detectron/ops/generate_proposal_labels.py
View file @ e5bb3a8f
......@@ -20,9 +20,9 @@ from __future__ import unicode_literals
import logging
from datasets import json_dataset
from utils import blob as blob_utils
import roi_data.fast_rcnn
from detectron.datasets import json_dataset
from detectron.utils import blob as blob_utils
import detectron.roi_data.fast_rcnn as fast_rcnn_roi_data
logger = logging.getLogger(__name__)
......@@ -40,13 +40,13 @@ class GenerateProposalLabelsOp(object):
roidb = blob_utils.deserialize(inputs[1].data)
im_info = inputs[2].data
im_scales = im_info[:, 2]
output_blob_names = roi_data.fast_rcnn.get_fast_rcnn_blob_names()
output_blob_names = fast_rcnn_roi_data.get_fast_rcnn_blob_names()
# For historical consistency with the original Faster R-CNN
# implementation we are *not* filtering crowd proposals.
# This choice should be investigated in the future (it likely does
# not matter).
json_dataset.add_proposals(roidb, rois, im_scales, crowd_thresh=0)
blobs = {k: [] for k in output_blob_names}
roi_data.fast_rcnn.add_fast_rcnn_blobs(blobs, im_scales, roidb)
fast_rcnn_roi_data.add_fast_rcnn_blobs(blobs, im_scales, roidb)
for i, k in enumerate(output_blob_names):
blob_utils.py_op_copy_blob(blobs[k], outputs[i])
lib/ops/generate_proposals.py → detectron/ops/generate_proposals.py
View file @ e5bb3a8f
......@@ -23,8 +23,8 @@
import numpy as np
from core.config import cfg
import utils.boxes as box_utils
from detectron.core.config import cfg
import detectron.utils.boxes as box_utils
class GenerateProposalsOp(object):
......
lib/roi_data/data_utils.py → detectron/roi_data/data_utils.py
View file @ e5bb3a8f
......@@ -26,9 +26,9 @@ import logging
import numpy as np
import threading
from core.config import cfg
from modeling.generate_anchors import generate_anchors
import utils.boxes as box_utils
from detectron.core.config import cfg
from detectron.modeling.generate_anchors import generate_anchors
import detectron.utils.boxes as box_utils
logger = logging.getLogger(__name__)
......
lib/roi_data/fast_rcnn.py → detectron/roi_data/fast_rcnn.py
View file @ e5bb3a8f
......@@ -27,12 +27,12 @@ import logging
import numpy as np
import numpy.random as npr
from core.config import cfg
import modeling.FPN as fpn
import roi_data.keypoint_rcnn
import roi_data.mask_rcnn
import utils.blob as blob_utils
import utils.boxes as box_utils
from detectron.core.config import cfg
import detectron.modeling.FPN as fpn
import detectron.roi_data.keypoint_rcnn as keypoint_rcnn_roi_data
import detectron.roi_data.mask_rcnn as mask_rcnn_roi_data
import detectron.utils.blob as blob_utils
import detectron.utils.boxes as box_utils
logger = logging.getLogger(__name__)
......@@ -124,7 +124,7 @@ def add_fast_rcnn_blobs(blobs, im_scales, roidb):
# all minibatch images
valid = True
if cfg.MODEL.KEYPOINTS_ON:
valid = roi_data.keypoint_rcnn.finalize_keypoint_minibatch(blobs, valid)
valid = keypoint_rcnn_roi_data.finalize_keypoint_minibatch(blobs, valid)
return valid
......@@ -203,13 +203,13 @@ def _sample_rois(roidb, im_scale, batch_idx):
# Optionally add Mask R-CNN blobs
if cfg.MODEL.MASK_ON:
roi_data.mask_rcnn.add_mask_rcnn_blobs(
mask_rcnn_roi_data.add_mask_rcnn_blobs(
blob_dict, sampled_boxes, roidb, im_scale, batch_idx
)
# Optionally add Keypoint R-CNN blobs
if cfg.MODEL.KEYPOINTS_ON:
roi_data.keypoint_rcnn.add_keypoint_rcnn_blobs(
keypoint_rcnn_roi_data.add_keypoint_rcnn_blobs(
blob_dict, roidb, fg_rois_per_image, fg_inds, im_scale, batch_idx
)
......
lib/roi_data/keypoint_rcnn.py → detectron/roi_data/keypoint_rcnn.py
View file @ e5bb3a8f
......@@ -27,9 +27,9 @@ from __future__ import unicode_literals
import logging
import numpy as np
from core.config import cfg
import utils.blob as blob_utils
import utils.keypoints as keypoint_utils
from detectron.core.config import cfg
import detectron.utils.blob as blob_utils
import detectron.utils.keypoints as keypoint_utils
logger = logging.getLogger(__name__)
......
lib/roi_data/loader.py → detectron/roi_data/loader.py
View file @ e5bb3a8f
......@@ -52,13 +52,13 @@ import uuid
from caffe2.python import core, workspace
from core.config import cfg
from roi_data.minibatch import get_minibatch
from roi_data.minibatch import get_minibatch_blob_names
from utils.coordinator import coordinated_get
from utils.coordinator import coordinated_put
from utils.coordinator import Coordinator
import utils.c2 as c2_utils
from detectron.core.config import cfg
from detectron.roi_data.minibatch import get_minibatch
from detectron.roi_data.minibatch import get_minibatch_blob_names
from detectron.utils.coordinator import coordinated_get
from detectron.utils.coordinator import coordinated_put
from detectron.utils.coordinator import Coordinator
import detectron.utils.c2 as c2_utils
logger = logging.getLogger(__name__)
......
lib/roi_data/mask_rcnn.py → detectron/roi_data/mask_rcnn.py
View file @ e5bb3a8f
......@@ -26,10 +26,10 @@ from __future__ import unicode_literals
import logging
import numpy as np
from core.config import cfg
import utils.blob as blob_utils
import utils.boxes as box_utils
import utils.segms as segm_utils
from detectron.core.config import cfg
import detectron.utils.blob as blob_utils
import detectron.utils.boxes as box_utils
import detectron.utils.segms as segm_utils
logger = logging.getLogger(__name__)
......
lib/roi_data/minibatch.py → detectron/roi_data/minibatch.py
View file @ e5bb3a8f
......@@ -32,11 +32,11 @@ import cv2
import logging
import numpy as np
from core.config import cfg
import roi_data.fast_rcnn
import roi_data.retinanet
import roi_data.rpn
import utils.blob as blob_utils
from detectron.core.config import cfg
import detectron.roi_data.fast_rcnn as fast_rcnn_roi_data
import detectron.roi_data.retinanet as retinanet_roi_data
import detectron.roi_data.rpn as rpn_roi_data
import detectron.utils.blob as blob_utils
logger = logging.getLogger(__name__)
......@@ -48,14 +48,14 @@ def get_minibatch_blob_names(is_training=True):
blob_names = ['data']
if cfg.RPN.RPN_ON:
# RPN-only or end-to-end Faster R-CNN
blob_names += roi_data.rpn.get_rpn_blob_names(is_training=is_training)
blob_names += rpn_roi_data.get_rpn_blob_names(is_training=is_training)
elif cfg.RETINANET.RETINANET_ON:
blob_names += roi_data.retinanet.get_retinanet_blob_names(
blob_names += retinanet_roi_data.get_retinanet_blob_names(
is_training=is_training
)
else:
# Fast R-CNN like models trained on precomputed proposals
blob_names += roi_data.fast_rcnn.get_fast_rcnn_blob_names(
blob_names += fast_rcnn_roi_data.get_fast_rcnn_blob_names(
is_training=is_training
)
return blob_names
......@@ -71,18 +71,18 @@ def get_minibatch(roidb):
blobs['data'] = im_blob
if cfg.RPN.RPN_ON:
# RPN-only or end-to-end Faster/Mask R-CNN
valid = roi_data.rpn.add_rpn_blobs(blobs, im_scales, roidb)
valid = rpn_roi_data.add_rpn_blobs(blobs, im_scales, roidb)
elif cfg.RETINANET.RETINANET_ON:
im_width, im_height = im_blob.shape[3], im_blob.shape[2]
# im_width, im_height corresponds to the network input: padded image
# (if needed) width and height. We pass it as input and slice the data
# accordingly so that we don't need to use SampleAsOp
valid = roi_data.retinanet.add_retinanet_blobs(
valid = retinanet_roi_data.add_retinanet_blobs(
blobs, im_scales, roidb, im_width, im_height
)
else:
# Fast R-CNN like models trained on precomputed proposals
valid = roi_data.fast_rcnn.add_fast_rcnn_blobs(blobs, im_scales, roidb)
valid = fast_rcnn_roi_data.add_fast_rcnn_blobs(blobs, im_scales, roidb)
return blobs, valid
......
lib/roi_data/retinanet.py → detectron/roi_data/retinanet.py
View file @ e5bb3a8f
......@@ -23,9 +23,9 @@ from __future__ import unicode_literals
import numpy as np
import logging
import utils.boxes as box_utils
import roi_data.data_utils as data_utils
from core.config import cfg
import detectron.utils.boxes as box_utils
import detectron.roi_data.data_utils as data_utils
from detectron.core.config import cfg
logger = logging.getLogger(__name__)
......
lib/roi_data/rpn.py → detectron/roi_data/rpn.py
View file @ e5bb3a8f
......@@ -24,10 +24,10 @@ import logging
import numpy as np
import numpy.random as npr
from core.config import cfg
import roi_data.data_utils as data_utils
import utils.blob as blob_utils
import utils.boxes as box_utils
from detectron.core.config import cfg
import detectron.roi_data.data_utils as data_utils
import detectron.utils.blob as blob_utils
import detectron.utils.boxes as box_utils
logger = logging.getLogger(__name__)
......
tests/data_loader_benchmark.py → detectron/tests/data_loader_benchmark.py
View file @ e5bb3a8f
......@@ -34,16 +34,18 @@ import pprint
import sys
import time
from caffe2.python import core, workspace, muji
from caffe2.python import core
from caffe2.python import muji
from caffe2.python import workspace
from core.config import assert_and_infer_cfg
from core.config import cfg
from core.config import merge_cfg_from_list
from core.config import merge_cfg_from_file
from datasets.roidb import combined_roidb_for_training
from roi_data.loader import RoIDataLoader
from utils.timer import Timer
import utils.logging
from detectron.core.config import assert_and_infer_cfg
from detectron.core.config import cfg
from detectron.core.config import merge_cfg_from_file
from detectron.core.config import merge_cfg_from_list
from detectron.datasets.roidb import combined_roidb_for_training
from detectron.roi_data.loader import RoIDataLoader
from detectron.utils.logging import setup_logging
from detectron.utils.timer import Timer
def parse_args():
......@@ -148,9 +150,9 @@ def main(opts):
if __name__ == '__main__':
workspace.GlobalInit(['caffe2', '--caffe2_log_level=0'])
logger = utils.logging.setup_logging(__name__)
logger = setup_logging(__name__)
logger.setLevel(logging.DEBUG)
logging.getLogger('roi_data.loader').setLevel(logging.INFO)
logging.getLogger('detectron.roi_data.loader').setLevel(logging.INFO)
np.random.seed(cfg.RNG_SEED)
args = parse_args()
logger.info('Called with args:')
......
tests/test_batch_permutation_op.py → detectron/tests/test_batch_permutation_op.py
View file @ e5bb3a8f
......@@ -26,8 +26,8 @@ from caffe2.python import core
from caffe2.python import gradient_checker
from caffe2.python import workspace
import utils.c2
import utils.logging
import detectron.utils.logging as logging_utils
import detectron.utils.c2 as c2_utils
class BatchPermutationOpTest(unittest.TestCase):
......@@ -105,7 +105,7 @@ class BatchPermutationOpTest(unittest.TestCase):
if __name__ == '__main__':
workspace.GlobalInit(['caffe2', '--caffe2_log_level=0'])
utils.c2.import_detectron_ops()
c2_utils.import_detectron_ops()
assert 'BatchPermutation' in workspace.RegisteredOperators()
utils.logging.setup_logging(__name__)
logging_utils.setup_logging(__name__)
unittest.main()
tests/test_bbox_transform.py → detectron/tests/test_bbox_transform.py
View file @ e5bb3a8f
......@@ -23,7 +23,7 @@ import unittest
from pycocotools import mask as COCOmask
import utils.boxes as box_utils
import detectron.utils.boxes as box_utils
def random_boxes(mean_box, stdev, N):
......
tests/test_cfg.py → detectron/tests/test_cfg.py
View file @ e5bb3a8f
......@@ -23,10 +23,10 @@ import tempfile
import unittest
import yaml
from core.config import cfg
from utils.collections import AttrDict
import core.config
import utils.logging
from detectron.core.config import cfg
from detectron.utils.collections import AttrDict
import detectron.core.config as core_config
import detectron.utils.logging as logging_utils
class TestAttrDict(unittest.TestCase):
......@@ -59,7 +59,7 @@ class TestAttrDict(unittest.TestCase):
# Serialize immutability state
a.immutable(True)
a2 = yaml.load(yaml.dump(a))
a2 = core_config.load_cfg(yaml.dump(a))
assert a.is_immutable()
assert a2.is_immutable()
......@@ -76,14 +76,14 @@ class TestCfg(unittest.TestCase):
s = 'dummy0'
cfg2 = copy.deepcopy(cfg)
cfg2.MODEL.TYPE = s
core.config.merge_cfg_from_cfg(cfg2)
core_config.merge_cfg_from_cfg(cfg2)
assert cfg.MODEL.TYPE == s
# Test: merge from yaml
s = 'dummy1'
cfg2 = yaml.load(yaml.dump(cfg))
cfg2 = core_config.load_cfg(yaml.dump(cfg))
cfg2.MODEL.TYPE = s
core.config.merge_cfg_from_cfg(cfg2)
core_config.merge_cfg_from_cfg(cfg2)
assert cfg.MODEL.TYPE == s
# Test: merge with a valid key
......@@ -91,7 +91,7 @@ class TestCfg(unittest.TestCase):
cfg2 = AttrDict()
cfg2.MODEL = AttrDict()
cfg2.MODEL.TYPE = s
core.config.merge_cfg_from_cfg(cfg2)
core_config.merge_cfg_from_cfg(cfg2)
assert cfg.MODEL.TYPE == s
# Test: merge with an invalid key
......@@ -100,13 +100,13 @@ class TestCfg(unittest.TestCase):
cfg2.FOO = AttrDict()
cfg2.FOO.BAR = s
with self.assertRaises(KeyError):
core.config.merge_cfg_from_cfg(cfg2)
core_config.merge_cfg_from_cfg(cfg2)
# Test: merge with converted type
cfg2 = AttrDict()
cfg2.TRAIN = AttrDict()
cfg2.TRAIN.SCALES = [1]
core.config.merge_cfg_from_cfg(cfg2)
core_config.merge_cfg_from_cfg(cfg2)
assert type(cfg.TRAIN.SCALES) is tuple
assert cfg.TRAIN.SCALES[0] == 1
......@@ -115,7 +115,7 @@ class TestCfg(unittest.TestCase):
cfg2.TRAIN = AttrDict()
cfg2.TRAIN.SCALES = 1
with self.assertRaises(ValueError):
core.config.merge_cfg_from_cfg(cfg2)
core_config.merge_cfg_from_cfg(cfg2)
def test_merge_cfg_from_file(self):
with tempfile.NamedTemporaryFile() as f:
......@@ -123,7 +123,7 @@ class TestCfg(unittest.TestCase):
s = cfg.MODEL.TYPE
cfg.MODEL.TYPE = 'dummy'
assert cfg.MODEL.TYPE != s
core.config.merge_cfg_from_file(f.name)
core_config.merge_cfg_from_file(f.name)
assert cfg.MODEL.TYPE == s
def test_merge_cfg_from_list(self):
......@@ -134,7 +134,7 @@ class TestCfg(unittest.TestCase):
assert cfg.TRAIN.SCALES[0] != 100
assert cfg.MODEL.TYPE != 'foobar'
assert cfg.NUM_GPUS != 2
core.config.merge_cfg_from_list(opts)
core_config.merge_cfg_from_list(opts)
assert type(cfg.TRAIN.SCALES) is tuple
assert len(cfg.TRAIN.SCALES) == 1
assert cfg.TRAIN.SCALES[0] == 100
......@@ -149,7 +149,7 @@ class TestCfg(unittest.TestCase):
_ = cfg.FINAL_MSG # noqa
with self.assertRaises(AttributeError):
_ = cfg.MODEL.DILATION # noqa
core.config.merge_cfg_from_list(opts)
core_config.merge_cfg_from_list(opts)
with self.assertRaises(AttributeError):
_ = cfg.FINAL_MSG # noqa
with self.assertRaises(AttributeError):
......@@ -164,7 +164,7 @@ class TestCfg(unittest.TestCase):
yaml.dump(cfg2, f)
with self.assertRaises(AttributeError):
_ = cfg.MODEL.DILATION # noqa
core.config.merge_cfg_from_file(f.name)
core_config.merge_cfg_from_file(f.name)
with self.assertRaises(AttributeError):
_ = cfg.MODEL.DILATION # noqa
......@@ -176,7 +176,7 @@ class TestCfg(unittest.TestCase):
with self.assertRaises(AttributeError):
_ = cfg.EXAMPLE.RENAMED.KEY # noqa
with self.assertRaises(KeyError):
core.config.merge_cfg_from_list(opts)
core_config.merge_cfg_from_list(opts)
def test_renamed_key_from_file(self):
# You should see logger messages like:
......@@ -191,9 +191,9 @@ class TestCfg(unittest.TestCase):
with self.assertRaises(AttributeError):
_ = cfg.EXAMPLE.RENAMED.KEY # noqa
with self.assertRaises(KeyError):
core.config.merge_cfg_from_file(f.name)
core_config.merge_cfg_from_file(f.name)
if __name__ == '__main__':
utils.logging.setup_logging(__name__)
logging_utils.setup_logging(__name__)
unittest.main()
tests/test_loader.py → detectron/tests/test_loader.py
View file @ e5bb3a8f
......@@ -24,10 +24,14 @@ import unittest
import mock
from caffe2.proto import caffe2_pb2
from caffe2.python import core, workspace, muji
from core.config import cfg, assert_and_infer_cfg
from roi_data.loader import RoIDataLoader
import utils.logging
from caffe2.python import core
from caffe2.python import muji
from caffe2.python import workspace
from detectron.core.config import assert_and_infer_cfg
from detectron.core.config import cfg
from detectron.roi_data.loader import RoIDataLoader
import detectron.utils.logging as logging_utils
def get_roidb_blobs(roidb):
......@@ -81,9 +85,14 @@ def run_net(net):
class TestRoIDataLoader(unittest.TestCase):
@mock.patch('roi_data.loader.get_minibatch_blob_names',
return_value=[u'data'])
@mock.patch('roi_data.loader.get_minibatch', side_effect=get_roidb_blobs)
@mock.patch(
'detectron.roi_data.loader.get_minibatch_blob_names',
return_value=[u'data']
)
@mock.patch(
'detectron.roi_data.loader.get_minibatch',
side_effect=get_roidb_blobs
)
def test_two_parallel_loaders(self, _1, _2):
train_data = np.random.rand(2, 3, 3).astype(np.float32)
train_loader, train_net = create_loader_and_network(train_data,
......@@ -102,9 +111,9 @@ class TestRoIDataLoader(unittest.TestCase):
if __name__ == '__main__':
workspace.GlobalInit(['caffe2', '--caffe2_log_level=0'])
logger = utils.logging.setup_logging(__name__)
logger = logging_utils.setup_logging(__name__)
logger.setLevel(logging.DEBUG)
logging.getLogger('roi_data.loader').setLevel(logging.INFO)
logging.getLogger('detectron.roi_data.loader').setLevel(logging.INFO)
np.random.seed(cfg.RNG_SEED)
cfg.TRAIN.ASPECT_GROUPING = False
cfg.NUM_GPUS = 2
......
tests/test_restore_checkpoint.py → detectron/tests/test_restore_checkpoint.py
View file @ e5bb3a8f
......@@ -26,16 +26,16 @@ import tempfile
from caffe2.python import workspace
from core.config import assert_and_infer_cfg
from core.config import cfg
from core.config import get_output_dir
from datasets.roidb import combined_roidb_for_training
from modeling import model_builder
import utils.c2 as c2_utils
import utils.logging
import utils.net as nu
from detectron.core.config import assert_and_infer_cfg
from detectron.core.config import cfg
from detectron.core.config import get_output_dir
from detectron.datasets.roidb import combined_roidb_for_training
from detectron.modeling import model_builder
from detectron.utils.logging import setup_logging
import detectron.utils.c2 as c2_utils
import detectron.utils.net as nu
utils.c2.import_detectron_ops()
c2_utils.import_detectron_ops()
def get_params(model):
......@@ -107,9 +107,9 @@ def test_restore_checkpoint():
if __name__ == '__main__':
workspace.GlobalInit(['caffe2', '--caffe2_log_level=0'])
logger = utils.logging.setup_logging(__name__)
logger = setup_logging(__name__)
logger.setLevel(logging.DEBUG)
logging.getLogger('roi_data.loader').setLevel(logging.INFO)
logging.getLogger('detectron.roi_data.loader').setLevel(logging.INFO)
np.random.seed(cfg.RNG_SEED)
output_dir = tempfile.mkdtemp()
# Generate config for test
......
tests/test_smooth_l1_loss_op.py → detectron/tests/test_smooth_l1_loss_op.py
View file @ e5bb3a8f
......@@ -26,8 +26,8 @@ from caffe2.python import core
from caffe2.python import gradient_checker
from caffe2.python import workspace
import utils.c2
import utils.logging
import detectron.utils.c2 as c2_utils
import detectron.utils.logging as logging_utils
class SmoothL1LossTest(unittest.TestCase):
......@@ -74,7 +74,7 @@ class SmoothL1LossTest(unittest.TestCase):
if __name__ == '__main__':
utils.c2.import_detectron_ops()
c2_utils.import_detectron_ops()
assert 'SmoothL1Loss' in workspace.RegisteredOperators()
utils.logging.setup_logging(__name__)
logging_utils.setup_logging(__name__)
unittest.main()
tests/test_spatial_narrow_as_op.py → detectron/tests/test_spatial_narrow_as_op.py
View file @ e5bb3a8f
......@@ -26,8 +26,8 @@ from caffe2.python import core
from caffe2.python import gradient_checker
from caffe2.python import workspace
import utils.c2
import utils.logging
import detectron.utils.c2 as c2_utils
import detectron.utils.logging as logging_utils
class SpatialNarrowAsOpTest(unittest.TestCase):
......@@ -85,7 +85,7 @@ class SpatialNarrowAsOpTest(unittest.TestCase):
if __name__ == '__main__':
workspace.GlobalInit(['caffe2', '--caffe2_log_level=0'])
utils.c2.import_detectron_ops()
c2_utils.import_detectron_ops()
assert 'SpatialNarrowAs' in workspace.RegisteredOperators()
utils.logging.setup_logging(__name__)
logging_utils.setup_logging(__name__)
unittest.main()
tests/test_zero_even_op.py → detectron/tests/test_zero_even_op.py
View file @ e5bb3a8f
......@@ -10,7 +10,7 @@ from caffe2.proto import caffe2_pb2
from caffe2.python import core
from caffe2.python import workspace
import utils.c2
import detectron.utils.c2 as c2_utils
class ZeroEvenOpTest(unittest.TestCase):
......@@ -107,6 +107,6 @@ class ZeroEvenOpTest(unittest.TestCase):
if __name__ == '__main__':
workspace.GlobalInit(['caffe2', '--caffe2_log_level=0'])
utils.c2.import_custom_ops()
c2_utils.import_custom_ops()
assert 'ZeroEven' in workspace.RegisteredOperators()
unittest.main()
lib/utils/blob.py → detectron/utils/blob.py
View file @ e5bb3a8f
......@@ -34,7 +34,7 @@ import numpy as np
from caffe2.proto import caffe2_pb2
from core.config import cfg
from detectron.core.config import cfg
def get_image_blob(im, target_scale, target_max_size):
......
lib/utils/boxes.py → detectron/utils/boxes.py
View file @ e5bb3a8f
......@@ -47,9 +47,9 @@ from __future__ import unicode_literals
import numpy as np
from core.config import cfg
import utils.cython_bbox as cython_bbox
import utils.cython_nms as cython_nms
from detectron.core.config import cfg
import detectron.utils.cython_bbox as cython_bbox
import detectron.utils.cython_nms as cython_nms
bbox_overlaps = cython_bbox.bbox_overlaps
......
lib/utils/c2.py → detectron/utils/c2.py
View file @ e5bb3a8f
......@@ -28,7 +28,7 @@ from caffe2.python import core
from caffe2.python import dyndep
from caffe2.python import scope
import utils.env as envu
import detectron.utils.env as envu
def import_contrib_ops():
......
lib/utils/env.py → detectron/utils/env.py
View file @ e5bb3a8f
......@@ -75,9 +75,10 @@ def get_detectron_ops_lib():
def get_custom_ops_lib():
"""Retrieve custom ops library."""
lib_dir, _utils = os.path.split(os.path.dirname(__file__))
det_dir, _ = os.path.split(os.path.dirname(__file__))
root_dir, _ = os.path.split(det_dir)
custom_ops_lib = os.path.join(
lib_dir, 'build/libcaffe2_detectron_custom_ops_gpu.so')
root_dir, 'build/libcaffe2_detectron_custom_ops_gpu.so')
assert os.path.exists(custom_ops_lib), \
'Custom ops lib not found at \'{}\''.format(custom_ops_lib)
return custom_ops_lib
lib/utils/keypoints.py → detectron/utils/keypoints.py
View file @ e5bb3a8f
......@@ -23,8 +23,8 @@ from __future__ import unicode_literals
import cv2
import numpy as np
from core.config import cfg
import utils.blob as blob_utils
from detectron.core.config import cfg
import detectron.utils.blob as blob_utils
def get_keypoints():
......
lib/utils/lr_policy.py → detectron/utils/lr_policy.py
View file @ e5bb3a8f
......@@ -22,7 +22,7 @@ from __future__ import unicode_literals
import numpy as np
from core.config import cfg
from detectron.core.config import cfg
def get_lr_at_iter(it):
......
lib/utils/net.py → detectron/utils/net.py
View file @ e5bb3a8f
......@@ -30,9 +30,11 @@ import yaml
from caffe2.python import core
from caffe2.python import workspace
from core.config import cfg
from utils.io import save_object
import utils.c2 as c2_utils
from detectron.core.config import cfg
from detectron.core.config import load_cfg
from detectron.utils.io import save_object
import detectron.utils.c2 as c2_utils
logger = logging.getLogger(__name__)
......@@ -59,7 +61,7 @@ def initialize_gpu_from_weights_file(model, weights_file, gpu_id=0):
with open(weights_file, 'r') as f:
src_blobs = pickle.load(f)
if 'cfg' in src_blobs:
saved_cfg = yaml.load(src_blobs['cfg'])
saved_cfg = load_cfg(src_blobs['cfg'])
configure_bbox_reg_weights(model, saved_cfg)
if 'blobs' in src_blobs:
# Backwards compat--dictionary used to be only blobs, now they are
......
lib/utils/subprocess.py → detectron/utils/subprocess.py
View file @ e5bb3a8f
......@@ -30,7 +30,7 @@ import subprocess
import cPickle as pickle
from six.moves import shlex_quote
from core.config import cfg
from detectron.core.config import cfg
import logging
logger = logging.getLogger(__name__)
......
lib/utils/train.py → detectron/utils/train.py
View file @ e5bb3a8f
......@@ -37,14 +37,14 @@ import re
from caffe2.python import memonger
from caffe2.python import workspace
from core.config import cfg
from core.config import get_output_dir
from datasets.roidb import combined_roidb_for_training
from modeling import model_builder
from utils import lr_policy
from utils.training_stats import TrainingStats
import utils.env as envu
import utils.net as nu
from detectron.core.config import cfg
from detectron.core.config import get_output_dir
from detectron.datasets.roidb import combined_roidb_for_training
from detectron.modeling import model_builder
from detectron.utils import lr_policy
from detectron.utils.training_stats import TrainingStats
import detectron.utils.env as envu
import detectron.utils.net as nu
def train_model():
......
lib/utils/training_stats.py → detectron/utils/training_stats.py
View file @ e5bb3a8f
......@@ -26,11 +26,12 @@ import datetime
import numpy as np
from caffe2.python import utils as c2_py_utils
from core.config import cfg
from utils.logging import log_json_stats
from utils.logging import SmoothedValue
from utils.timer import Timer
import utils.net as nu
from detectron.core.config import cfg
from detectron.utils.logging import log_json_stats
from detectron.utils.logging import SmoothedValue
from detectron.utils.timer import Timer
import detectron.utils.net as nu
class TrainingStats(object):
......
lib/utils/vis.py → detectron/utils/vis.py
View file @ e5bb3a8f
......@@ -26,9 +26,9 @@ import os
import pycocotools.mask as mask_util
from utils.colormap import colormap
import utils.env as envu
import utils.keypoints as keypoint_utils
from detectron.utils.colormap import colormap
import detectron.utils.env as envu
import detectron.utils.keypoints as keypoint_utils
# Matplotlib requires certain adjustments in some environments
# Must happen before importing matplotlib
......
docker/Dockerfile
View file @ e5bb3a8f
This diff is collapsed.
lib/setup.py → setup.py
View file @ e5bb3a8f
This diff is collapsed.
tools/convert_pkl_to_pb.py
View file @ e5bb3a8f
This diff is collapsed.
tools/convert_selective_search.py
View file @ e5bb3a8f
This diff is collapsed.
tools/generate_testdev_from_test.py
View file @ e5bb3a8f
This diff is collapsed.
tools/infer.py
View file @ e5bb3a8f
This diff is collapsed.
tools/infer_simple.py
View file @ e5bb3a8f
This diff is collapsed.
tools/reval.py
View file @ e5bb3a8f
This diff is collapsed.
tools/test_net.py
View file @ e5bb3a8f
This diff is collapsed.
tools/train_net.py
View file @ e5bb3a8f
This diff is collapsed.
tools/visualize_results.py
View file @ e5bb3a8f
This diff is collapsed.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment