How to check celery for bugs

Does it need checking? Yes. It is mostly infested, in both Israel and the USA. (Rav Moshe Vaya) Celery is highly infested.

USA	★★	mostly infested (muchzak b’negiut)
Israel	★★	mostly infested (muchzak b’negiut)

Rav Moshe Vaya, Bedikat HaMazon K’Halacha, Degree of Infestation chart, p. 937 (Celery)

Bugs sit on the celery and need to be washed out. His video does not name the type of bug.
OU Kosher’s chart names them: thrips, flies or worms, on the inside and outside of the stalk.

Take off all the leaves. Everything has to be cleaned.
Once everything is exposed, wash it well under the faucet with a strong stream of water.
While it is under the water, brush it with a brush or something else, to get out all the bugs.
If there are cracks or holes in the stalks, open them and see what is there.
Do all this before Shabbat or Yom Tov. Do not clean it on Shabbat, because you are killing the bugs.
If you forgot and it is already Shabbat or Yom Tov (e.g. the night of Pesach), the only thing you can do is cut off the outer layer, like peeling it.

Rav Moshe Vaya, Bedikat HaMazon K’Halacha, Degree of Infestation chart, p. 937 (rating).
OU Kosher: thrips, flies or worms, on the inside and outside of the stalk, especially near the base, or on the leaves. Remove all leaves. Hold the celery under a strong stream of water and brush down the inside and outside of the stalk with a finger or a vegetable brush. If a worm burrow is seen, slit the stalk open, remove the worm and wash well. Leaves must be washed with soapy solution. (Fruit and Vegetable Inspection Chart)
cRc: non-organic stalks: cut away and discard the leaves, then wash each stalk under a stream of water. Organic stalks: discard the leaves, put the stalks in a bowl of soapy water, then wash each stalk under a stream of water. Organic celery hearts: put them in soapy water and agitate lightly, then wash each stalk front and back under running water. (Fruits & Vegetables Checking Guide)

References: OU Kosher — Fruit and Vegetable Inspection Chart (PDF) · cRc — Fruits & Vegetables Checking Guide