Thursday, December 11, 2014

NP Problems Tutorial with Coding Examples [By Solving a CodeJam Problem]

What is NP?

NP is the set of all decision problems (question with yes-or-no answer) for which the 'yes'-answers can be verified in polynomial time (O(n^k) where n is the problem size, and k is a constant) by a deterministic Turing machine. Polynomial time is sometimes used as the definition of fast or quickly.
--------------------------------------------

What is P?

P is the set of all decision problems which can be solved in polynomial time by a deterministic Turing machine. Since it can solve in polynomial time, it can also be verified in polynomial time. Therefore P is a subset of NP.
--------------------------------------------

What is NP-Complete?

A problem x that is in NP is also in NP-Complete if and only if every other problem in NP can be quickly (ie. in polynomial time) transformed into x. In other words:
  1. x is in NP, and
  2. Every problem in NP is reducible to x
So what makes NP-Complete so interesting is that if any one of the NP-Complete problems was to be solved quickly then all NP problems can be solved quickly. Also see What’s “P=NP?”, and why is it such a famous question?
--------------------------------------------

What is NP-Hard?

NP-Hard are problems that are at least as hard as the hardest problems in NP. Note that NP-Complete problems are also NP-hard. However not all NP-hard problems are NP (or even a decision problem), despite having 'NP' as a prefix. That is the NP in NP-hard does not mean 'non-deterministic polynomial time'. Yes this is confusing but its usage is entrenched and unlikely to change.

Photo From Wikipedia [1]

--------------------------------------------

P Problem Coding Example:

In Google's CodeJam Africa 2010, Qualification Round, there were a problem called Store Credit. Here the Description of it:
  • Problem: You receive a credit C at a local store and would like to buy two items. You first walk through the store and create a list L of all available items. From this list you would like to buy two items that add up to the entire value of the credit. The solution you provide will consist of the two integers indicating the positions of the items in your list (smaller number first).
  • Input: The first line of input gives the number of cases, N. N test cases follow. For each test case there will be:
    • One line containing the value C, the amount of credit you have at the store.
    • One line containing the value I, the number of items in the store.
    • One line containing a space separated list of I integers. Each integer P indicates the price of an item in the store.
    • Each test case will have exactly one solution.
  • Output: For each test case, output one line containing "Case #x: " followed by the indices of the two items whose price adds up to the store credit. The lower index should be output first. Limits: 5 ≤ C ≤ 1000, 1 ≤ P ≤ 1000Small dataset: N = 10, 3 ≤ I ≤ 100Large dataset: N = 50, 3 ≤ I ≤ 2000
  • Sample: 
Input: 
3
100 
3 
5 75 25 
200 
7 
150 24 79 50 88 345 3
8
8
2 1 9 4 4 56 90 3
Output: 
Case #1: 2 3 
Case #2: 1 4 
Case #3: 4 5
    • C++ Solution:
    #include <iostream>     //For std::vector
    #include <fstream>      //For std::cout and std::cin
    #include <vector>       //For std::vector

    using namespace std;

    int main(int argc, char ** argv)
    {
        std::ifstream infile(argv[1]);
        if(!infile) {
            std::cout << "could not open " << infile << std::endl;
            return 0;
        }
        std::ofstream outfile(argv[2]);
        if(!outfile) {
            std::cout << "could not open " << outfile << std::endl;
            return 0;
        }
       
        /*should be char if c, but with C++ this
        will let "infile >>" to read only one digit*/
        unsigned short numberOf_testCases;
        unsigned short credit, numberOf_Items, itemPrice;
        std::vector<unsigned short> allItems;
       
        infile >> numberOf_testCases;    //Number of Test Cases
        for(unsigned int i=0; i<numberOf_testCases; i++){
            infile >> credit;            //Credit
            infile >> numberOf_Items;    //Number of Items
            for(unsigned int j=0; j<numberOf_Items; j++){
                infile >> itemPrice;
                allItems.push_back(itemPrice);
            }
           
            bool breakBool = false;
            for(unsigned int j=0; j<allItems.size(); j++){
                for(unsigned int k=j+1; k<allItems.size(); k++){
                   if ((allItems[j] + allItems[k]) == credit)
                   {
                     outfile << "Case #" << i+1;
                     outfile << ": " << j+1 << " " << k+1 << endl;
                     breakBool = true;
                     break;
                   }
                }
                if (breakBool)
                    break;
            }
            allItems.clear();
        }
       
        return 0;   
    }
    --------------------------------------------

    NP Problem Coding Example :

    The previous example problem is not an NP problem. The previous solution implementation is of order O(n^2) at worst case.
    Now, we will modify the previous problem slightly to make it an NP complete problem (A subset sum problem).
    • Problem Modification: From the item list you have created for the store, you should buy items that you can add up to the entire value of the credit.
      You will buy any number of items not only two.
    • Input Modification: No modifications.
    • Output Modification: No Modifications.
    • Sample: 
    Input: 
    2 
    100 
    3 
    5 75 25
    8 
    5
    4 2 1 1 5 
    Output: 
    Case #1: 2 3 
    Case #2: 1 2 3 4
    • C++ Solution: This solution is using a dynamic programming (a bottom-up approach).
    #include <iostream>
    #include <fstream>
    #include <algorithm>    // std::reverse
    #include <vector>       // std::vector
    using namespace std;
    std::vector<unsigned short> getIndecesOfSum(
        std::vector<unsigned short> numbers, int n, unsigned short sum);
    int main(int argc, char ** argv)
    {
        std::ifstream infile(argv[1]);
        if(!infile) {
            std::cout << "could not open " << infile << std::endl;
            return 0;
        }
        std::ofstream outfile(argv[2]);
        if(!outfile) {
            std::cout << "could not open " << outfile << std::endl;
            return 0;
        }
       
        /*should be char if c, but with C++ this  
        will let "infile >>" to read only one digit*/
        unsigned short numberOf_testCases; 
        unsigned short credit, numberOf_Items, itemPrice;
        std::vector<unsigned short> allItems, selectedItems;
       
        infile >> numberOf_testCases;    //Number of Test Cases
        for(unsigned int i=0; i<numberOf_testCases; i++){
            infile >> credit;            //Credit
            infile >> numberOf_Items;    //Number of Items
            for(unsigned int j=0; j<numberOf_Items; j++){
                infile >> itemPrice;
                allItems.push_back(itemPrice);
            }
           
            selectedItems = getIndecesOfSum(allItems, 
                                       allItems.size(), credit);
            outfile << "Case #" << i+1 << ": ";
            for(unsigned int j=0; j<selectedItems.size(); j++){
                outfile << selectedItems[j] << " ";
            }
            outfile << endl;
            allItems.clear();
            selectedItems.clear();
        }
       
        return 0;   
    }
    std::vector<unsigned short> getIndecesOfSum(
          std::vector<unsigned short> numbers, int n, unsigned short sum)
    {
        std::vector<unsigned short> IndecesOfSum;
        bool DP_Matrix[sum+1][n+1]; //Dynamic Programming Matrix
        // If sum is 0, then answer is true
        for (int i = 0; i <= n; i++)
            DP_Matrix[0][i] = true;
        // If sum is not 0 and numbers is empty, then answer is false
        for (int i = 1; i <= sum; i++)
            DP_Matrix[i][0] = false;
        // Fill the DP_Matrix table in bottom up manner
        for (int i = 1; i <= sum; i++)
        {
            for (int j = 1; j <= n; j++) //Row then Row
            {
              /*If DP Matrix element is true,
              then next elements in the row is true.*/
              DP_Matrix[i][j] = DP_Matrix[i][j-1]; 
              if (i >= numbers[j-1])
               DP_Matrix[i][j] = DP_Matrix[i][j] || 
                                  DP_Matrix[i - numbers[j-1]][j-1];
            }
        }
        //Print the Dynamic Programming Matrix
        /*for (int i = 0; i <= sum; i++)
        {
            for (int j = 0; j <= n; j++)
                printf ("%4d", DP_Matrix[i][j]);
            printf("\n");
        }*/
       
        for (int j = n; j >= 1; j--)
        {
            if(sum - numbers[j-1] >= 0 && 
                    DP_Matrix[ sum - numbers[j-1] ][j-1])
            {
                IndecesOfSum.push_back(j);
                sum = sum - numbers[j-1];
            }
        }
       
        std::reverse(IndecesOfSum.begin(), IndecesOfSum.end());
       
        //return DP_Matrix[sum][n];
        return IndecesOfSum;
    }

    --------------------------------------------

    A More Difficult Problem:

    Now we are going to make this problem more difficult.
    • Problem Modification: From the item list you have created for the store, you should list all the combinations of items that you can add up to the entire value of the credit. You will buy any number of items not only two.
    • Input Modification: No modifications.
    • Output Modification: The program should list all the possible combinations of items that you can bought with the received credit C. Now there is no unique solution for each test case.For each test case, output one line containing "Case #x: " followed by the indices of the two items whose price adds up to the store credit for the first solution. If there are other solutions, it should be also written after it and be separated with a comma ",".
    • Sample: 
    Input: 
    2 
    100
    3 
    5 75 25
    8 
    5
    4 2 4 1 1
    Output: 
    Case #1: 2 3 
    Case #2: 1 2 4 5 ,  1 3 , 2 3 4 5
    • C++ Solution: Think about it. I will post the answer in a coming post though ;-)
    • A Question: Now the problem is more difficult, but does it still an NP Complete problem ? Wait for the answer in the coming post.
    --------------------------------------------

    To compile and use the above codes:

    1. Add it to a file: D:\Test_Project\main.cpp,
    2. then make a new patch file: D:\Test_Project\Compile.bat:, with the content:
      g++.exe -Wall -c -g main.cpp -o main.oppg++.exe -static -static-libgcc -static-libstdc++ -o "program.exe" main.opppause
    3. Run (Double Click) that patch command to build your executable D:\Test_Project\program.exe.The "pause" command is to hold the CMD window for you to check if the C++ compilation was successful or not.Don't forget to install the a C++ compiler, for the patch file to work correctly.If you are using windows you can install the GNU Compiler by downloading and installing MinGW (Minimalist GNU for Windows).
    4. Open CMD, then write the following command:D:\Test_Project\program A-large-practice.in A-large-practice.outDon't forget to put the file D:\Test_Project\A-large-practice.in with the input test cases, and to create a new empty file D:\Test_Project\A-large-practice.out.
    5. Check the results into D:\Test_Project\A-large-practice.out.

    --------------------------------------------

    References:

    Wednesday, December 10, 2014

    Tuesday, December 9, 2014

    Measure Real Time with TRACE32 (Lauterbach Debugger)

    Measuring Real Time for a microcontroller using a Lauterbach Debugger and TRACE32 Software is very easy.
    In the following image example, The execution time of the function UpdateCGI is measured:

    1. Put a breakpoint in the line of its call and another one at the line just after it.
    2. Open the RunTime tool from the Misc menu.
    3. Run the code, the debugger will stop at the first breakpoint.
    4. Then run again, so that it stops at the second breakpoint.
    5. Read the execution time from the RunTime window (laststart time).

    Note:
    This is assuming that, between the two breakpoints, no other code from another task is executed.
    So, to guarantee the accuracy of your results,
    it's preferred to either put the code you want to measure (code inside UpdateCGI in the above example) into a critical section or
    grammatically stop the hardware timer at the start of that code and resume it again at the end of the code.

    a[3] == 3[a] : True, False, or Syntax Error?

    In C, if we have an array a :
    unsigned char a[5]; 

    Is the following expression :
    a[3] == 3[a]
    true or false, or its a syntax Error ?

    It's true because of Pointer Arithmetic.
    This is the direct artifact of arrays behaving as pointers, "a" is a memory address. "a[3]" is the value that's 3 elements further from "a". The address of this element is "a + 3".
    This is equal to offset "
    a" from "3" elements at the beginning of the address space "3 + a".

    Simpler, the C standard defines the [] operator as follows:
    a[b] == *(a + b)
    Therefore a[3] will evaluate to:
    *(a + 3)
    and 3[a] will evaluate to:
    *(3 + a)
    and this is why those are equal.

    Wednesday, December 3, 2014

    Polymorphism Simply !

    If you think about the Greek roots of the term "Polymorphism", it should become obvious.
    • Poly = many: polygon = many-sided, polystyrene = many styrenes (a), polyglot = many languages, and so on.
    • Morph = change or form: morphology = study of biological form, Morpheus = the Greek god of dreams able to take any form.
    So polymorphism is the ability (in programming) to present the same interface for differing underlying forms (data types).

    If, for example, you have two classes: DogPet and CatPet that inherits form a parent class: Pet, like:

    class Pet
    {
        public virtual string GetPetSound()
        {
            return "Each Animal has its own different Sound.";
        }
    }

    class DogPet : Pet
    {
        public override string GetPetSound()
        {
            return "Woof!";
        }
    }

    class CatPet : Pet
    {
        public override string GetPetSound()
        {
            return "Meaw!";
        }
    }

    And you have defined the following instances and called a member function: GetPetSound, like:

    Pet dog = new DogPet();
    Pet cat = new CatPet();

    string dog_sound = dog.GetPetSound();
    string cat_sound = cat.GetPetSound();

    Each individual type of pet can figure out the details on their own, allowing the code that uses it to focus on other things. This is how the Virtual keyword is related to Polymorphism.

    Method Overloading is not a part of Polymorphism. However Method Overriding is indeed part of Polymorphism, as we did in the above example.

    Some people consider Method Overloading a Compile Time Polymorphism where there are functions with same name and different parameters.
    They also classify Method Overriding as a Run-Time Polymorphism where there are Functions in the extended class with same name and same parameters as in the base class, but with different behaviors.

    Method Overloading Example:
    public class A
    {
        public void print(int x, int y)
        {
            Console.WriteLine("Parent Method");
        }
    }
    
    public class B : A
    {
        public void child()
        {
            Console.WriteLine("Child Method");
        }
    
        public void print(float x, float y)
        {
            Console.WriteLine("Overload child method");
        }
    }
    Method Overriding Example:
    public class A
    {
        public virtual void print()
        {
            Console.WriteLine("Parent Method");
        }
    }
    
    public class B : A
    {
        public void child()
        {
            Console.WriteLine("Child Method");
        }
    
        public override void print()
        {
            Console.WriteLine("Overriding child method");
        }
    }

    Branch Prediction Penalty

    Consider the following piece of code that calculates the checksum of a selection from a large array of data:

    UCHAR u8Data[40000];
    for (USHORT i = 0; i < 40000; i++)
    {  
      u8Data[i] = std::rand() % 256;
    } 
    UCHAR  u8Checksum = (UCHAR)0;
    for (USHORT j = 0; j < 40000; j++){  
      if (u8Data[j] >= 128)
      {    
        u8Checksum += u8Data[j];
      }
    }

    You can consider the if condition “if (u8Data[j] >= 128)” as a railroad junction just as in the following picture:


    Suppose that the microprocessor is the operator of that junction and it hears a train coming and have no idea which way it will go. It will stop the train to ask the captain which direction he wants, and then sets the switch appropriately. Then it will starts up the heavy train that have a lot of momentum once again, which will consume a lot of time. This is why microprocessors don’t work in such way.

    A more intelligent operator of that junction would ‘guess’ which direction the train will go. And if he suggested right, the train continues on. If he suggested wrong, he will stop the train and reallocate the junction. On the average, this technique will save him half of the time he was wasting without ‘guessing’. This is why modern microprocessors use what is called Branch Prediction techniques.

    Modern microprocessors are complicated and have long pipelines. So they take a long time to "warm up" and "slow down". That if condition, at the processor level, is a branch instruction like the following:

    Photo by [1]

    When the microprocessor sees that branch instruction, it has no idea which way it will go until it evaluates the branch condition. Evaluating the condition of the branch is not decided in the moment of executing the branch instruction itself because of using pipelining. The microprocessor can act like the non-intelligent railroad junction operator and halts execution and waits until the previous instructions are completed. Then he continues down the correct path.
    But is there a better way? Yes, there is, the microprocessor guesses which direction the branch will go before evaluating the branch condition.
    • If it guessed right, it continues executing, and no time is wasted.
    • If it guessed wrong, it needs to flush the pipeline and roll back to the branch. Then it can restart down the other path.
    Going back to our original piece of code, you may be surprised to know that you can speed its execution by around 6 times just by adding the following line of code before the second for loop:

        std::sort(u8Data, u8Data + 40000);

    With a sorted array, the condition “u8Data[j] >= 128” is first false for a streak of values, then becomes true for all later values. That's easy to predict for the microprocessor. With an unsorted array, you pay all the branching costs.


    References: 
    1- http://stackoverflow.com/questions/11227809
    2- http://en.wikipedia.org/wiki/Branch_predication