I think this is because I have the USB B to PC set to send Master. I didn't try assigning the USB B to PC to specific channels because that wouldn't serve your purpose.
"USB has a strict "tree" topology and "master-slave" protocol for addressing peripheral devices; peripheral devices cannot interact with one another except via the host, and two hosts cannot communicate over their USB ports directly." -- Source: Wikipedia USB
This looks like you are connecting the slave (T4S) to two masters (recorder and backing tracks).