Summary:
Pull Request resolved: https://github.com/facebookresearch/Detectron/pull/857

Move GenreateProposalsOp on GPU for detectron

Reviewed By: rbgirshick

Differential Revision: D14779322

fbshipit-source-id: 3034e33ac06c227a33946063233f8626c0a442be

Summary:
OpenCV 2, 3, 4 have different signatures for this function. Using [-2] is compatible with them.
Pull Request resolved: https://github.com/facebookresearch/Detectron/pull/847

Reviewed By: rbgirshick

Differential Revision: D14703716

Pulled By: ir413

fbshipit-source-id: 7329c5d45a0b0d2a3125198e6ac64bc79b42ff46

Summary:
The current/old example does not link to the correct subdirectories.
The old example created two links to the same parent directory instead of two links to the seperate subdirectories.
I corrected the example code, so the two links link to the two seperate subdirectories.
Pull Request resolved: https://github.com/facebookresearch/Detectron/pull/846

Reviewed By: rbgirshick

Differential Revision: D14703901

Pulled By: ir413

fbshipit-source-id: b6525b7414a8b9c2dba819a76bae245d06a3b499

Summary:
This adds an exponential learning rate schedule, parameterized by `GAMMA` which specifies the ratio between the final and initial learning rate.  This is a more natural parameterization than `A ^ t` or `exp(A * t)`.

Since most jobs drop by ~3 orders of magnitude throughout training, typically `GAMMA = 1e-3` should work for many scenarios.

Reviewed By: ashwinb, rbgirshick

Differential Revision: D14654419

fbshipit-source-id: 8dabbc8df28d32e4fd748b61bc32efaf3935d244

Summary:
This adds a cosine schedule, based on:
 - "Bag of Freebies for Training Object Detection Neural Networks" https://arxiv.org/pdf/1902.04103.pdf
  - "SGDR: Stochastic Gradient Descent with Warm Restarts" https://arxiv.org/pdf/1608.03983.pdf

Reviewed By: ashwinb

Differential Revision: D14593944

fbshipit-source-id: f0f7eedee784fa4d0aebd092d718526bbf627d81

Summary:
When saving models, it can appear as available for loading before saving has finished.  This can cause race conditions, particularly on network file systems.

This changes saving code to write to a temporary filename in the same directory and then atomically rename to the target filename.  This is a common technique that relies on the atomicity of `os.rename`.

Reviewed By: ashwinb, samialsheikh

Differential Revision: D14495989

fbshipit-source-id: 2e44dd8e747331b319ef46132ddeff604618a5dd

Summary:
It is convenient to run with a set of precomputed proposals for a
larger set (e.g., coco_trainval2014) and then do experiments using annotations
for a smaller subset of the data.

It is possible to do this filtering only if a config flag is set, but it seems
harmless enough to do it always.

Reviewed By: rbgirshick

Differential Revision: D14106953

fbshipit-source-id: f726425ee5b6b22fcb711e3ec479b302ea09e837

Reviewed By: rbgirshick

Differential Revision: D13546589

fbshipit-source-id: eb0a442a4620fa4d82a57671cabb58cdb5ed02df

Summary:
Per t34270005 we're changing all publicly hosted detectron assets to a new low-cost CDN-backed S3 bucket. The migration requires rewriting all instances of `https://s3-us-west-2.amazonaws.com/detectron` to `https://dl.fbaipublicfiles.com/detectron`.

This giant codemod does it.

```
cd fbcode

rg -l s3-us-west-2.amazonaws.com/detectron > ~/filelist.txt

for F in $(cat ~/filelist.txt);
  sed -i '' -e 's/s3-us-west-2\.amazonaws\.com\/detectron/dl.fbaipublicfiles.com\/detectron/g' $F
done
```

Reviewed By: jma127

Differential Revision: D13538734

fbshipit-source-id: 4fe5c797770b0d9f675cee53dbcb6b52857fa45d

Summary:
Codemod generated with clangr shard mode, 50 files per diff,
clangr code(ndim()->dim()): diffusion/FBS/browse/master/fbcode/caffe2/caffe2/fb/codemods/TensorMethodRename.cpp

Reviewed By: ezyang

Differential Revision: D12935918

fbshipit-source-id: 4ce24a3debcd471b3649fa71f890f7b24bb8678a

Reviewed By: rbgirshick

Differential Revision: D10857403

fbshipit-source-id: 5ea53ece3418e8f6dcb20df3d3521d661a7380a3

Summary:
Codemod generated with clangr shard mode, 50 files per diff,
clangr code(size->numel): diffusion/FBS/browse/master/fbcode/caffe2/caffe2/fb/codemods/TensorMethodRename.cpp

Reviewed By: ezyang

Differential Revision: D12833487

fbshipit-source-id: 5dec5a64f91912620f3a5e48a2cc571be675c5fd

Reviewed By: ashwinb

Differential Revision: D10496498

fbshipit-source-id: eb12fe573ec3270172e27c2fdb39a70fc92d8d99

Reviewed By: rbgirshick

Differential Revision: D10508713

fbshipit-source-id: c410d88cc64864087dacd04a73639952cb063df0

Reviewed By: cdelahousse

Differential Revision: D10452557

fbshipit-source-id: d64c10e7b0c6ecce2db50c6470ae282ac76db9a0

Reviewed By: rbgirshick

Differential Revision: D10452054

fbshipit-source-id: ee5c909844c034dc2e4a1ee995f3c86e1f353694

Reviewed By: rbgirshick

Differential Revision: D10433948

fbshipit-source-id: 0d4dbc7f532e13c559da65c403aa06238f612e25

Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/12456

codemod with 'Yes to all'
codemod -d . --extensions h,cc,cpp,cu,py,proto,pbtxt,pb.txt,config cuda_gpu_id device_id

Overload TextFormat::ParseFromString to do string replace when parsing from protobuf format

Reviewed By: Yangqing

Differential Revision: D10240535

fbshipit-source-id: 5e6992bec961214be8dbe26f16f5794154a22b25

Reviewed By: ashwinb

Differential Revision: D10223705

fbshipit-source-id: bb46ffea8185abb7abf6df7273c2941c6d95f8b1

Reviewed By: ashwinb

Differential Revision: D10223289

fbshipit-source-id: 1e6fb55f77e01a2f653fd614d13d1d4a7ceeccea

Reviewed By: ashwinb

Differential Revision: D10218305

fbshipit-source-id: d9d285959b1e4f5c0d38f080d6ce8c260b1959e0

Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/12232

Original commit changeset: fca91fea58b7

This adds proper modifications to the DeviceType <->DeviceOption conversion code added in D10033396

Reviewed By: jerryzh168

Differential Revision: D10132473

fbshipit-source-id: 801ef777e2950982cb47b48051b1471a0a91e64b

Reviewed By: rbgirshick

Differential Revision: D10131443

fbshipit-source-id: ee6df3147b94711eecbe558bcaf30d021976b593

Differential Revision:
D10123245

Original commit changeset: d83da8e00a12

fbshipit-source-id: fca91fea58b7df208edc2e218a1d514f9821ec7b

Reviewed By: rbgirshick

Differential Revision: D9691247

fbshipit-source-id: 7a9c1a33698be6907dede589965f8f4f4d337f93

Summary:
A couple small issues pointed out by Ross.

 - Treating bytes as strings with a subprocess' output :/
 - Controling precision of logged floats using a workaround instead of
   FLOAT_REPR which doesn't work with newer versions of Python3.

Reviewed By: rbgirshick

Differential Revision: D9724292

fbshipit-source-id: a6aa1730f25df5d165291dc30b9350a9fff6fca6

Summary:
There were two "bugs" associated with loading pickle files:

The first is easy: these files should have been `open()`ed as binary, but
weren't.

The second is slightly nuanced. The default encoding used wwhile unpickling
is 7-bit (ASCII.) However, the blobs are arbitrary 8-bit bytes which don't
agree. The absolute correct way to do this is to use `encoding="bytes"` and
then interpret the blob names either as ASCII, or better, as unicode utf-8. A
reasonable fix, however, is to treat it the encoding as 8-bit latin1 (which
agrees with the first 256 characters of Unicode anyway.)

As part of this, I also centralized all pickling operations into
`detectron.utils.io`.

This /still/ does not change the build to Python3, but I believe it is ready
now.

Reviewed By: rbgirshick

Differential Revision: D9689294

fbshipit-source-id: add1f2d784fe196df27b20e65e35922536d11a3c

Summary:
This is a first step towards python3 compatibility. Specifically, it
tackles:

 - `cPickle`: uses six.moves
 - `Queue`: uses six.moves
 - `urllib2`: uses six.moves and changes `urllib2.urlopen` to `urllib.request.urlopen`

Also, fundamentally it changes the types of all config "byte" types to
"string" types. Those configurations aren't un-encoded byte streams but very
specifically ascii (or unicode encoded) strings which are specified and
consumed by human eyes.

Reviewed By: rbgirshick

Differential Revision: D9662024

fbshipit-source-id: b8372f685b57ec4260ae881a2f8bb7967f337b10

Summary:
Original commit changeset: f5614a5d2607

D9986213 is causing Multifeed Aggregator a [huge performance different](https://our.intern.facebook.com/intern/ads/analyze_canary/412951953278781781/) and is blocking aggregator push since last Friday night: https://fburl.com/feedtools/b6izvwjz
We need to land this revert ASAP to unblock aggregator push.

Reviewed By: orionr

Differential Revision: D10123245

fbshipit-source-id: d83da8e00a1250f5d09811a0a587c127e377aab2

Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/12022

codemod -d . --extensions h,cc,cpp,cu,py,proto,pbtxt,pb.txt,config cuda_gpu_id device_id

codemod with 'Yes to all'

Reviewed By: orionr

Differential Revision: D9986213

fbshipit-source-id: f5614a5d26078817aee8caf79a494abfd6a95ff1

Summary:
Soft-NMS is present in test.py, but it is missing in test_retinanet.py.
Pull Request resolved: https://github.com/facebookresearch/Detectron/pull/670

Reviewed By: rbgirshick

Differential Revision: D10035825

Pulled By: ir413

fbshipit-source-id: c49ac7e595b32d2ac757815df4add1c644a34027

Summary:
Fixing error:
TypeError: Layout of the output array img is incompatible with cv::Mat (step[ndims-1] != elemsize or step[1] != elemsize*nchannels)

in vis_class(..) and vis_bbox(..) methods when calling vis_one_image_opencv(..).
Pull Request resolved: https://github.com/facebookresearch/Detectron/pull/555

Reviewed By: rbgirshick

Differential Revision: D10027949

Pulled By: ir413

fbshipit-source-id: e34b8b881ffe231984f270910f778c5abe79e489

Summary:
Detectron code is inconsistent about when it uses (1, 1, 1, 1) as the
regression weights vs. `cfg.MODEL.BBOX_REG_WEIGHTS` (which has a default
value of (10, 10, 5, 5)). This change allows the caller to make the call
(heh!) about what to use.

Reviewed By: rbgirshick

Differential Revision: D9981875

fbshipit-source-id: 8798408bd2592486eb1a2e40ccb70b9bedfdbf46

Reviewed By: rbgirshick

Differential Revision: D9940697

fbshipit-source-id: 60a7d76a6ac2ef89a58f6c3dca8adce27ffd74f7

Reviewed By: rbgirshick

Differential Revision: D9818825

fbshipit-source-id: a1f2182a7760568987100aa7402aca9327dee952

Summary:
Negative labels were never set if `len(bg_inds) <= num_bg`. This
condition happens when there aren't as many negative anchor samples compared to the
typical batch size (256). Since detection scales are typically ~800, there are
plenty of negative samples and this bug is likely never hit. In any case, it
is certainly a bug which should be fixed!

Reviewed By: rbgirshick

Differential Revision: D9647703

fbshipit-source-id: fada9f4f2e0a0b5c1619940328c505e73f9a1c7e