test-lib.sh: do not barf under --debug at the end of the test

test-lib.sh: do not barf under --debug at the end of the test

The original did "does $remove_trash exist?  Then go one level above
and remove it".  There was no problem under "--debug", where
the variable is left empty, as the first "test -d $remove_trash" would
have said "No, it doesn't".

With the check implemented in the previous step, we'd always get an
error under "--debug".

Noticed-by: Jeff King <redacted>
Signed-off-by: Junio C Hamano <redacted>