A logic gate takes one or two bits in and gives one bit out. Gates are built from transistors, tiny electronic switches; a modern processor has billions of them. Everything a computer does is built from gates.
NOT flips its input: 0 becomes 1 and 1 becomes 0. AND gives 1 only if both inputs are 1. OR gives 1 if at least one input is 1. XOR (exclusive or) gives 1 if the inputs are different.
A B | AND OR XOR
0 0 | 0 0 0
0 1 | 0 1 1
1 0 | 0 1 1
1 1 | 1 1 0
That table is called a truth table: every possible input, with the output for each. NAND (NOT AND) is special: every other gate can be built out of NAND gates alone.
Gates can do arithmetic. Adding two bits gives a sum bit and a carry: 1 + 1 is 10 in binary, sum 0, carry 1. Look at the table: the sum is exactly XOR, and the carry is exactly AND. That pair of gates is a half adder. Chain full adders (which also take a carry in) side by side, and you can add numbers of any length. That's what's in the arithmetic part of a processor.