InputFormatInfo
Related Docs:
class InputFormatInfo
| package scheduler
Computes the preferred locations based on input(s) and returned a location to block map.
Computes the preferred locations based on input(s) and returned a location to block map. Typical use of this method for allocation would follow some algo like this:
a) For each host, count number of splits hosted on that host. b) Decrement the currently allocated containers on that host. c) Compute rack info for each host and update rack to count map based on (b). d) Allocate nodes based on (c) e) On the allocation result, ensure that we don't allocate "too many" jobs on a single node (even if data locality on that is very high) : this is to prevent fragility of job if a single (or small set of) hosts go down.
go to (a) until required nodes are allocated.
If a node 'dies', follow same procedure.
PS: I know the wording here is weird, hopefully it makes some sense !