Combine chicken, 2 teaspoons salt and 2 quarts water in a Dutch oven.
Cover, bring to a boil. Reduce heat to medium, add pepper, onion and celery leaves.
Simmer chicken until it falls away from the bone.
Remove the chicken and onion from the pot and strain. Discard onion and celery leaves. Set chicken aside to cool.
When cooled, remove chicken from bones and cut into 1" pieces. Set aside. Discard skin and bones, raise heat to high and bring broth to a boil.
Combine flour, baking soda and the remaining 1/2 teaspoon salt in a medium bowl.
Cut in shortening until mixture resembles coarse crumbs. Gradually stir in milk until moist.
Turn dough out onto a floured surface and lightly knead about 5 minutes.
Roll out to 1/2" thickness and cut into 1 1/2" squares.
Drop dough into the boiling broth.
Reduce heat to low, cover and cook for 10 minutes. Stir gently. Add chicken back to the pot and stir until it is heated through.