Big Data – Hadoop Ecosystems
Import the accounts table into HDFS file system:
1) Import account:
$ sqoop import \
–connect jdbc:mysql://localhost/loudacre \
–username training –password training \
–table accounts \
–target-dir /loudacre/accounts \
–null-non-string ‘\\N’
2) List the contents of the accounts directory:
$ hdfs dfs -ls /loudacre/accounts
3) Import incremental updates to accounts
As Loudacre adds new accounts in MySQL accounts table, the account data in HDFS must be updated as accounts are created. You can use Sqoop to append these new records.
Run the add_new_accounts.py script to add the latest accounts to MySQL.
$ DEV1/exercises/sqoop/add_new_accounts.py
Incrementally import and append the newly added accounts to the accounts directory. Use Sqoop to import on the last value on the acct_num column largest account ID:
$ sqoop import \
–connect jdbc:mysql://localhost/loudacre \
–username training –password training \
–incremental append \
–null-non-string ‘\\N’ \
–table accounts \
–target-dir /loudacre/accounts \
–check-column acct_num \
–last-value <largest_acct_num>
4) You should see three new files. Use Hadoop’s cat command to view the entire contents of these files.
hdfs dfs -cat /loudacre/accounts/part-m-0000[456]
Delivering a high-quality product at a reasonable price is not enough anymore.
That’s why we have developed 5 beneficial guarantees that will make your experience with our service enjoyable, easy, and safe.
You have to be 100% sure of the quality of your product to give a money-back guarantee. This describes us perfectly. Make sure that this guarantee is totally transparent.
Read moreEach paper is composed from scratch, according to your instructions. It is then checked by our plagiarism-detection software. There is no gap where plagiarism could squeeze in.
Read moreThanks to our free revisions, there is no way for you to be unsatisfied. We will work on your paper until you are completely happy with the result.
Read moreYour email is safe, as we store it according to international data protection rules. Your bank details are secure, as we use only reliable payment systems.
Read moreBy sending us your money, you buy the service we provide. Check out our terms and conditions if you prefer business talks to be laid out in official language.
Read more