Important paper from Google on large batch optimization. They do impressively careful experiments measuring # iterations needed to achieve target validation error at various batch sizes. The main "surprise" is the lack of surprises. [thread]
Most Liked Replies
for the interesting review. Their use of data parallelism +
the SGD variants particularly those without/with
Nesterov momentum (optimization-wise) caught my
interest so reviewed it shortly after it appeared at
You May Also Like
In that spirit, here's some quick Things Many People Find Too Obvious To Have Told You Already.
Your idea is not valuable, at all. All value is in the execution. You think you are an exception; you are not. You should not insist on an NDA to talk about it; nobody serious will engage in contract review over an idea, and this will mark you as clueless.
Technologists tend to severely underestimate the difficulty and expense of creating software, especially at companies which do not have fully staffed industry leading engineering teams ("because software is so easy there, amirite guys?")
Charge more. Charge more still. Go on.
The press is a lossy and biased compression of events in the actual world, and is singularly consumed with its own rituals, status games, and incentives. The news necessarily fails to capture almost everything which happened yesterday. What it says is important usually isn't.
Companies find it incredibly hard to reliably staff positions with hard-working generalists who operate autonomously and have high risk tolerances. This is not the modal employee, including at places which are justifiably proud of the skill/diligence/etc of their employees.
Seek wealth, not money or status. Wealth is having assets that earn while you sleep. Money is how we transfer time and wealth. Status is your place in the social hierarchy.
Understand that ethical wealth creation is possible. If you secretly despise wealth, it will elude you.
Ignore people playing status games. They gain status by attacking people playing wealth creation games.
You’re not going to get rich renting out your time. You must own equity - a piece of a business - to gain your financial freedom.
2/ Sales is often viewed as either a saving grace or proof that the product isn’t good enough (because it should sell itself). Neither are ever true. Some common mistakes that result in...
3/ Mistake 1: Hire a sales rep before reaching product/market fit to get your initial batch of customers. This is a mistake because founders need to work through their MVP with early adopters to truly understand what it is they’re selling.
4/ Mistake 2: Reach product/market fit, need to scale, and rely entirely on self-serve. For enterprise products that require big commitments and internal shifts, almost no product is self-explanatory enough to sell itself.
5/ Mistake 3: Make a first sales hire who isn’t scrappy enough to help mold the sales process from scratch. Some salespeople are amazing at their jobs, but not cut out to establish the processes that others end up following. This skillset is what @rdedatta calls a “sales ninja”.
Important paper from Google on large batch optimization. They do impressively careful experiments measuring # iterations needed to achieve target validation error at various batch sizes. The main "surprise" is the lack of surprises. [thread]https://t.co/7QIx5CFdfJ pic.twitter.com/rYWst5SOmY— Roger Grosse (@RogerGrosse) November 24, 2018