Apache Hadoop Developer Certification Questions (108 Q&As)

Are you preparing for the Apache Hadoop Developer Exam?Are you looking for study materials about Apache Hadoop Developer test? You can download the Apache Hadoop Developer pdf demo on PassQuestion web site freely.Then make up your mind you can pass the Apache Hadoop Developer Exam easily by using our Apache Hadoop Developer Certification Questions.Let us help you climb that ladder of success and pass your Hadoop 2.0 Certification exam for Pig and Hive Developer exam now!

Test Online Apache Hadoop Developer Free Questions

1. Which one of the following statements describes a Pig bag. tuple, and map, respectively?

Unordered collection of maps, ordered collection of tuples, ordered set of key/value pairs

Unordered collection of tuples, ordered set of fields, set of key value pairs

Ordered set of fields, ordered collection of tuples, ordered collection of maps

Ordered collection of maps, ordered collection of bags, and unordered set of key/value pairs

2. You want to run Hadoop jobs on your development workstation for testing before you submit them to your production cluster.

Which mode of operation in Hadoop allows you to most closely simulate a production cluster while using a single machine?

Run all the nodes in your production cluster as virtual machines on your development workstation.

Run the hadoop command with the Cjt local and the Cfs file:///options.

Run the DataNode, TaskTracker, NameNode and JobTracker daemons on a single machine.

Run simldooop, the Apache open-source software for simulating Hadoop clusters.

3. Which HDFS command uploads a local file X into an existing HDFS directory Y?

hadoop scp X Y

hadoop fs -localPut X Y

hadoop fs-put X Y

hadoop fs -get X Y

4. In Hadoop 2.0, which TWO of the following processes work together to provide automatic failover of the NameNode? Choose 2 answers

ZKFailoverController

ZooKeeper

QuorumManager

JournalNode

5. To use a lava user-defined function (UDF) with Pig what must you do?

Define an alias to shorten the function name

Pass arguments to the constructor of UDFs implementation class

Put the JAR file into the user's home folder in HDFS

6. When is the earliest point at which the reduce method of a given Reducer can be called?

As soon as at least one mapper has finished processing its input split.

As soon as a mapper has emitted at least one record.

Not until all mappers have finished processing all records.

It depends on the InputFormat used for the job.

7. Which one of the following statements describes the relationship between the ResourceManager and the ApplicationMaster?

The ApplicationMaster requests resources from the ResourceManager

The ApplicationMaster starts a single instance of the ResourceManager

The ResourceManager monitors and restarts any failed Containers of the ApplicationMaster

The ApplicationMaster starts an instance of the ResourceManager within each Container

8. Which HDFS command copies an HDFS file named foo to the local filesystem as localFoo?

hadoop fs -get foo LocalFoo

hadoop -cp foo LocalFoo

hadoop fs -Is foo

hadoop fs -put foo LocalFoo

9. You need to perform statistical analysis in your MapReduce job and would like to call methods in the Apache Commons Math library, which is distributed as a 1.3 megabyte Java archive (JAR) file.

Which is the best way to make this library available to your MapReducer job at runtime?

Have your system administrator copy the JAR to all nodes in the cluster and set its location in the HADOOP_CLASSPATH environment variable before you submit your job.

Have your system administrator place the JAR file on a Web server accessible to all cluster nodes and then set the HTTP_JAR_URL environment variable to its location.

When submitting the job on the command line, specify the Clibjars option followed by the JAR file path.

Package your code and the Apache Commands Math library into a zip file named JobJar.zip

10. In a MapReduce job with 500 map tasks, how many map task attempts will there be?

It depends on the number of reduces in the job.

Between 500 and 1000.

At most 500.

At least 500.

Exactly 500.

11. You want to count the number of occurrences for each unique word in the supplied input data. You’ve decided to implement this by having your mapper tokenize each word and emit a literal value 1, and then have your reducer increment a counter for each literal 1 it receives. After successful implementing this, it occurs to you that you could optimize this by specifying a combiner.

Will you be able to reuse your existing Reduces as your combiner in this case and why or why not?

Yes, because the sum operation is both associative and commutative and the input and output types to the reduce method match.

No, because the sum operation in the reducer is incompatible with the operation of a Combiner.

No, because the Reducer and Combiner are separate interfaces.

No, because the Combiner is incompatible with a mapper which doesn’t use the same data type for both the key and value.

Yes, because Java is a polymorphic object-oriented language and thus reducer code can be reused as a combiner.

12. What data does a Reducer reduce method process?

All the data in a single input file.

All data produced by a single mapper.

All data for a given key, regardless of which mapper(s) produced it.

All data for a given value, regardless of which mapper(s) produced it.

13. Given a directory of files with the following structure: line number, tab character, string:

Example:

1abialkjfjkaoasdfjksdlkjhqweroij

2kadfjhuwqounahagtnbvaswslmnbfgy

3kjfteiomndscxeqalkzhtopedkfsikj

You want to send each line as one record to your Mapper.

Which InputFormat should you use to complete the line: conf.setInputFormat (____.class) ; ?

SequenceFileAsTextInputFormat

SequenceFileInputFormat

KeyValueFileInputFormat

BDBInputFormat

14. Examine the following Hive statements:

Assuming the statements above execute successfully, which one of the following statements is true?

Each reducer generates a file sorted by age

The SORT BY command causes only one reducer to be used

The output of each reducer is only the age column

The output is guaranteed to be a single file with all the data sorted by age

15. When can a reduce class also serve as a combiner without affecting the output of a MapReduce program?

When the types of the reduce operation’s input key and input value match the types of the reducer’s output key and output value and when the reduce operation is both communicative and associative.

When the signature of the reduce method matches the signature of the combine method.

Always. Code can be reused in Java since it is a polymorphic object-oriented programming language.

Always. The point of a combiner is to serve as a mini-reducer directly after the map phase to increase performance.

Never. Combiners and reducers must be implemented separately because they serve different purposes.

16. What does the following WebHDFS command do?

Curl -1 -L “http://host:port/webhdfs/v1/foo/bar?op=OPEN”

Make a directory /foo/bar

Read a file /foo/bar

List a directory /foo

Delete a directory /foo/bar

17. You need to run the same job many times with minor variations. Rather than hardcoding all job configuration options in your drive code, you’ve decided to have your Driver subclass org.apache.hadoop.conf.Configured and implement the org.apache.hadoop.util.Tool interface.

Indentify which invocation correctly passes.mapred.job.name with a value of Example to Hadoop?

hadoop “mapred.job.name=Example” MyDriver input output

hadoop MyDriver mapred.job.name=Example input output

hadoop MyDrive CD mapred.job.name=Example input output

hadoop setproperty mapred.job.name=Example MyDriver input output

hadoop setproperty (“mapred.job.name=Example”) MyDriver input output

18. Determine which best describes when the reduce method is first called in a MapReduce job?

Reducers start copying intermediate key-value pairs from each Mapper as soon as it has completed. The programmer can configure in the job what percentage of the intermediate data should arrive before the reduce method begins.

Reducers start copying intermediate key-value pairs from each Mapper as soon as it has completed. The reduce method is called only after all intermediate data has been copied and sorted.

Reduce methods and map methods all start at the beginning of a job, in order to provide optimal performance for map-only or reduce-only jobs.

Reducers start copying intermediate key-value pairs from each Mapper as soon as it has completed. The reduce method is called as soon as the intermediate key-value pairs start to arrive.

19. You have a directory named jobdata in HDFS that contains four files: _first.txt, second.txt, .third.txt and #data.txt.

How many files will be processed by the FileInputFormat.setInputPaths () command when it’s given a path object representing this directory?

Four, all files will be processed

Three, the pound sign is an invalid character for HDFS file names

Two, file names with a leading period or underscore are ignored

None, the directory cannot be named jobdata

One, no special characters can prefix the name of an input file

20. In a large MapReduce job with m mappers and n reducers, how many distinct copy operations will there be in the sort/shuffle phase?

mXn (i.e., m multiplied by n)

m+n (i.e., m plus n)

mn (i.e., m to the power of n)

21. Which Hadoop component is responsible for managing the distributed file system metadata?

NameNode

Metanode

DataNode

NameSpaceManager

22. Review the following data and Pig code.

M,38,95111

F,29,95060

F,45,95192

M,62,95102

F,56,95102

A = LOAD 'data' USING PigStorage('.') as (gender:Chararray, age:int, zlp:chararray);

B = FOREACH A GENERATE age;

Which one of the following commands would save the results of B to a folder in hdfs named myoutput?

STORE A INTO 'myoutput' USING PigStorage(',');

DUMP B using PigStorage('myoutput');

STORE B INTO 'myoutput';

DUMP B INTO 'myoutput';

23. MapReduce v2 (MRv2/YARN) splits which major functions of the JobTracker into separate daemons? Select two.

Heath states checks (heartbeats)

Resource management

Job scheduling/monitoring

Job coordination between the ResourceManager and NodeManager

Launching tasks

Managing file system metadata

MapReduce metric reporting

Managing tasks

24. Assuming the following Hive query executes successfully:

Which one of the following statements describes the result set?

A bigram of the top 80 sentences that contain the substring "you are" in the lines column of the input data A1 table.

An 80-value ngram of sentences that contain the words "you" or "are" in the lines column of the inputdata table.

A trigram of the top 80 sentences that contain "you are" followed by a null space in the lines column of the inputdata table.

A frequency distribution of the top 80 words that follow the subsequence "you are" in the lines column of the inputdata table.

25. Given the following Pig commands:

Which one of the following statements is true?

The $1 variable represents the first column of data in ‘my.log’

The $1 variable represents the second column of data in ‘my.log’

The severe relation is not valid

The grouped relation is not valid

26. What does Pig provide to the overall Hadoop solution?

Legacy language Integration with MapReduce framework

Simple scripting language for writing MapReduce programs

Database table and storage management services

C++ interface to MapReduce and data warehouse infrastructure

27. What types of algorithms are difficult to express in MapReduce v1 (MRv1)?

Algorithms that require applying the same mathematical function to large numbers of individual binary records.

Relational operations on large amounts of structured and semi-structured data.

Algorithms that require global, sharing states.

Large-scale graph algorithms that require one-step link traversal.

Text analysis algorithms on large collections of unstructured text (e.g, Web crawls).

28. You need to create a job that does frequency analysis on input data. You will do this by writing a Mapper that uses TextInputFormat and splits each value (a line of text from an input file) into individual characters. For each one of these characters, you will emit the character as a key and an InputWritable as the value.

As this will produce proportionally more intermediate data than input data, which two resources should you expect to be bottlenecks?

Processor and network I/O

Disk I/O and network I/O

Processor and RAM

Processor and disk I/O

29. Which one of the following statements regarding the components of YARN is FALSE?

A Container executes a specific task as assigned by the ApplicationMaster

The ResourceManager is responsible for scheduling and allocating resources

A client application submits a YARW job to the ResourceManager

The ResourceManager monitors and restarts any failed Containers

30. You are developing a combiner that takes as input Text keys, IntWritable values, and emits Text keys, IntWritable values.

Which interface should your class implement?

Combiner <Text, IntWritable, Text, IntWritable>

Mapper <Text, IntWritable, Text, IntWritable>

Reducer <Text, Text, IntWritable, IntWritable>

Reducer <Text, IntWritable, Text, IntWritable>

Combiner <Text, Text, IntWritable, IntWritable>

31. Which one of the following Hive commands uses an HCatalog table named x?

SELECT * FROM x;

SELECT x.-FROM org.apache.hcatalog.hive.HCatLoader(‘x’);

SELECT * FROM org.apache.hcatalog.hive.HCatLoader(‘x’);

Hive commands cannot reference an HCatalog table

32. Given the following Pig command:

logevents = LOAD 'input/my.log' AS (date:chararray, levehstring, code:int, message:string);

Which one of the following statements is true?

The logevents relation represents the data from the my.log file, using a comma as the parsing delimiter

The logevents relation represents the data from the my.log file, using a tab as the parsing delimiter

The first field of logevents must be a properly-formatted date string or table return an error

The statement is not a valid Pig command

33. Consider the following two relations, A and B.

C = DOIN B BY a1, A by b2;

C = JOIN A by al, B by b2;

C = JOIN A a1, B b2;

C = JOIN A SO, B $1;

34. Given the following Hive commands:

Which one of the following statements Is true?

The file mydata.txt is copied to a subfolder of /apps/hive/warehouse

The file mydata.txt is moved to a subfolder of /apps/hive/warehouse

The file mydata.txt is copied into Hive’s underlying relational database 0.

The file mydata.txt does not move from Its current location in HDFS

35. In a MapReduce job, the reducer receives all values associated with same key.

Which statement best describes the ordering of these values?

The values are in sorted order.

The values are arbitrarily ordered, and the ordering may vary from run to run of the same MapReduce job.

The values are arbitrary ordered, but multiple runs of the same MapReduce job will always have the same ordering.

Since the values come from mapper outputs, the reducers will receive contiguous sections of sorted values.

36. Which describes how a client reads a file from HDFS?

The client queries the NameNode for the block location(s). The NameNode returns the block location(s) to the client. The client reads the data directory off the DataNode(s).

The client queries all DataNodes in parallel. The DataNode that contains the requested data responds directly to the client. The client reads the data directly off the DataNode.

The client contacts the NameNode for the block location(s). The NameNode then queries the DataNodes for block locations. The DataNodes respond to the NameNode, and the NameNode redirects the client to the DataNode that holds the requested data block(s). The client then reads the data directly off the DataNode.

The client contacts the NameNode for the block location(s). The NameNode contacts the DataNode that holds the requested data block. Data is transferred from the DataNode to the NameNode, and then from the NameNode to the client.

37. For each input key-value pair, mappers can emit:

As many intermediate key-value pairs as designed. There are no restrictions on the types of those key-value pairs (i.e., they can be heterogeneous).

As many intermediate key-value pairs as designed, but they cannot be of the same type as the input key-value pair.

One intermediate key-value pair, of a different type.

One intermediate key-value pair, but of the same type.

As many intermediate key-value pairs as designed, as long as all the keys have the same types and all the values have the same type.

38. You write MapReduce job to process 100 files in HDFS. Your MapReduce algorithm uses TextInputFormat: the mapper applies a regular expression over input values and emits key-values pairs with the key consisting of the matching text, and the value containing the filename and byte offset. Determine the difference between setting the number of reduces to one and settings the number of reducers to zero.

There is no difference in output between the two settings.

With zero reducers, no reducer runs and the job throws an exception. With one reducer, instances of matching patterns are stored in a single file on HDF

With zero reducers, all instances of matching patterns are gathered together in one file on HDF

With one reducer, instances of matching patterns are stored in multiple files on HDF

With zero reducers, instances of matching patterns are stored in multiple files on HDF

With one reducer, all instances of matching patterns are gathered together in one file on HDF

39. In Hadoop 2.0, which one of the following statements is true about a standby NameNode?

The Standby NameNode:

Communicates directly with the active NameNode to maintain the state of the active NameNode.

Receives the same block reports as the active NameNode.

Runs on the same machine and shares the memory of the active NameNode.

Processes all client requests and block reports from the appropriate DataNodes.

40. In the reducer, the MapReduce API provides you with an iterator over Writable values.

What does calling the next () method return?

It returns a reference to a different Writable object time.

It returns a reference to a Writable object from an object pool.

It returns a reference to the same Writable object each time, but populated with different data.

It returns a reference to a Writable object. The API leaves unspecified whether this is a reused object or a new object.

It returns a reference to the same Writable object if the next value is the same as the previous value, or a new Writable object otherwise.

Question 1 of 40

Apache Hadoop Developer Certification Questions (108 Q&As)

Test Online Apache Hadoop Developer Free Questions

Leave a Reply Cancel reply