Limits and Order
For functions of a real variable, the derivative is defined as
f′(x)=
lim
∆x→ 0
f(x+∆x) − f(x)
∆x
,
which means that the difference
f(x+∆x) − f(x)
∆x
− f′(x)
is small if ∆x is small and not 0 (for which the
quotient is not obviously defined).
Multiplying the remainder by ∆x, we obtain that
f(x+∆x) − f(x)−f′(x) ∆x= small·∆x,
with the right hand side (RHS) of the equation is "much smaller than
∆x" in the precise sense
lim
∆x→ 0
RHS
|∆x|
=0.
Another formal advantage is that the equation is also defined and
true for ∆x
= 0.
Definition. An expression (function) ϕ(x) is little o of
x as x → 0, written ϕ(x) = o(x) [as x → 0], if
lim
x → 0
ϕ(x)
|x|
= 0.
If we are not worried about the particular details of ϕ(x),
we write ϕ(x) = o(x) [as x → 0].
With this convention, the
definition of differentiability and the derivative takes the
convenient form
f(x+∆x) = f(x) +f′(x)·∆x+ o(∆x).
In a similar way, if limx → 0 ψ(x) = 0, we write
ψ(x) = o(1) with the precise meaning that
lim
x→ 0
ψ(x)
1
= 0.
Definition. Let q(x) be nonzero for x near and not equal 0. Then a function ϕ(x) is little o of q(x), written ϕ(x)=o(q(x)), if
lim
x → 0
ϕ(x)
|q(x)|
= 0.
Then a function ϕ(x) is big O of q(x), written ϕ(x)=O(q(x)), if
ϕ(x)
|q(x)|
is bounded as x → 0.
N.B. We are assuming that, for x small
enough and ≠ 0, both ϕ(x) and q(x) are defined and
q(x) ≠ 0. If q(x) might possibly be 0 for some x near
0, we also propose an ϵ−δ definition of
little o(q):
Definition. A function f = o(q) as x → 0 means For every ϵ > 0, there is a δ > 0 such that if 0 < |x| < δ, then |f(x)| ≤ ϵ|q(x)|
With this convention, continuity of a function f(x) can be
expressed by
f(x+∆x) = f(x) + o(1)
as ∆x→ 0.
A real valued function of a real variable x is differentiable
at x with derivative f′(x) if
f(x+∆x) = f(x) +f′(x) ∆x+ o(∆x)
as ∆x→ 0.
Local boundedness of a function can be expressed as
f(x+∆x)=O(1)
as ∆x→ 0.
There is a formal calculus for handling sums and products for
functions which are little o or big O of one (or
several) q. Verify that O(1) ·o(∆x) = o(∆x); i.e., the product of a bounded function and a
function which is o(∆x) is o(∆x).
Similarly o(∆x) ±o(∆x) = o(∆x).
Proof of the Chain RuleThe Chain Rule. Let g(z) be differentiable at z, and let
f(w) be differentiable at w = g(z). Then h(z) = f(g(z)) is differentiable at
z and
h′(z) =
d
dz
f(g(z)) = f′(g(z))·g′(z).
Proof: We show that
h(z + ∆z)
= f(g(z + ∆z))
= f(g(z)) + f′(g(z)) ·g′(z)·∆z+ o(∆z).
as ∆z→ 0.
Let
∆g(z) = g(z + ∆z) − g(z) = g′(z) ∆z+ o(∆z).
We are assuming that
g(z+∆z)
= g(z) +g′(z) ·∆z+ o(∆z),
f(g(z)+∆g(z))
= f(g(z)) +f′(g(z)) ·∆g(z) +o(∆g(z)).
Since
∆g(z)
= g′(z) ∆z+o(∆z)
= O(∆z),
o(∆g(z))
= o(O(∆z))
= o(∆z),
we have
f(g(z + ∆z))
= f(g(z)) +f′(g(z)) ·g′(z) ·∆z+ o(∆z).
Remarks
The concepts little o and big O are also useful as the
argument x → ∞. For example we write
x2=o(ex) as x→ ∞ with the precise meaning
lim
x→∞
x2
ex
=0.
The concepts little o and big O are also useful
for infinite limits.
For example we write
ln|x|=o(1/x) as x→ 0 with the precise meaning
lim
x→ 0
ln|x|
1/x
=0.
File translated from
TEX
by
TTH,
version 4.03. On 05 Mar 2012, 09:14.