DSet<'U>

Creates a new dataset whose elements are the results of applying the given function to each of the elements of the dataset. The given function is an async function, in which elements within the same collection are parallelly executed with Async.Parallel

Creates a new dataset whose elements are the results of applying the given function to each of the elements of the dataset. The given function is an async function, in which elements within the same collection are parallelly executed with Async.Parallel. The first integer index passed to the function indicates the partition, and the second integer passed to the function index (from 0) element within the partition.

Bin sort the DSet. Apply a partition function, repartition elements of dataset across nodes in the cluster. The number of partitions remains unchanged. Elements within each partition/bin are sorted using the 'comparer'.

Bin sort the DSet. Apply a partition function, repartition elements of dataset into 'numPartitions" partitions across nodes in the cluster. Elements within each partition/bin are sorted using the 'comparer'.

Bin sort the DSet. Apply a partition function, repartition elements of dataset across nodes in the cluster according to the setting specified by "param". Elements within each partition/bin are sorted using the 'comparer'.

Bypass the Dset in 2 ways. One of the DSet is pulled, the other DSet will be in push dataflow.

Bypass the Dset in 2 ways. One of the DSet is pulled, the other DSet will be in push dataflow.

Bypass the Dset in 3 ways. One of the DSet is pulled, the other two DSets will be in push dataflow.

Bypass the Dset in 4 ways. One of the DSet is pulled, the other three DSets will be in push dataflow.

Bypass the Dset in n ways. One of the DSet is pulled, the other n - 1 DSets will be in push dataflow.

Cache DSet content in memory, in raw object form.

Applies the given function to each element of the dataset. The new dataset comprised of the results for each element where the function returns Some with some value.

Applies the given function to each element of the dataset and concatenates all the results.

Count the number of elements(rows) in the DSet

Cross Join, for each entry in x:Dset<'U> and x1:Dset<'U1> apply mapFunc:'U->'U1->'U2 The resultant set is the product of Dset<'U> and Dset<'U1>

Cross Join with Filtering, for each entry in x:Dset<'U> and x1:Dset<'U1> apply mapFunc:'U->'U1->'U2 option. Filter out all those result that returns None.

Cross Join with Fold, for each entry in x:DSet<'U> and x1:DSet<'U1> apply mapFunc:'U->'U1->'U2, and foldFunc: 'S->'U2->'S

Span a sequence (dataset) to a distributed dataset by splitting the sequence to N partitions, with N being the number of nodes in the cluster. Each node get one partition of the dataset. The number of partitions of the distributed dataset is N, which is the number of the active nodes in the cluster.

Span a sequence (dataset) to a distributed dataset by splitting the sequence to NUM * N partitions, with NUM being the number of partitions on each node, and N being the number of nodes in the cluster. Each node get NUM partition of the dataset.

Execute a distributed customerized functional delegate on each of the machine. The functional delegate is running remotely on each node. If any node in the cluster is not responding, the functional delegate does not run on that node.

Execute num distributed customerized functional delegates on each of the machine. The functional delegate is running remotely on each node. If any node in the cluster is not responding, the functional delegates do not run on that node.

Creates a new dataset containing only the elements of the dataset for which the given predicate returns true.

Fold the entire DSet with a fold function, an aggregation function, and an initial state. The initial state is deserialized (separately) for each partition. Within each partition, the elements are folded into the state variable using 'folder' function. Then 'aggrFunc' is used to aggregate the resulting state variables from all partitions to a single state.

Get number of partitions. If partition has not been set, setup the partition

Identity Mapping, the new DSet is the same as the old DSet, with an encode function installed.

Generate a distributed dataset by importing a customerized functional delegate from a local contract store. The imported functional delegate is usually exported by another service (either in the same remote container or in anotherPrajna remote container. Each node runs one local instance of the functional delegate, which forms one partition of the dataset. The number of partitions of dataset is N, where N is the number of the nodes in the cluster. If any node in the cluster is not responding, the dataset does not contain the data resultant from the functional delegates in that node.

Generate a distributed dataset by importing customerized functional delegates from a local contract store. The imported functional delegate is usually exported by another service (either in the same remote container or in anotherPrajna remote container. Each node runs multiple local instance of the functional delegates, each of which forms one partition of the dataset. The number of partitions of dataset is N * num , where N is the number of the nodes in the cluster, and num is the number of contracts in the contract list. If any node in the cluster is not responding, the dataset does not contain the data resultant from the functional delegates in that node.

Create a distributed dataset on the distributed cluster, with each element created by a functional delegate.

Create a distributed dataset on the distributed cluster, with each element created by a functional delegate, using a given number of parallel execution per node. The functional delegate that create each element in the dataset, the integer index passed to the function indicates the partition, and the second integer passed to the function index (from 0) element within the partition. The functional delegate that returns the size of the partition, the integer index passed to the function indicates the partition.

Create a distributed dataset on the distributed cluster, with each element created by a functional delegate.

Applies a function 'iterFunc' to each element

Lazily save the DSet to HDD. This is a lazy evaluation. This DSet must be a branch generated by Bypass or a child of such branch. The save action will be triggered when a pull happens on other branches generated by Bypass.

Lazily save the DSet to HDD using "name". This is a lazy evaluation. This DSet must be a branch generated by Bypass or a child of such branch. The save action will be triggered when a pull happens on other branches generated by Bypass.

Trigger to load metadata.

Read DSet back to local machine and apply a function on each value. Caution: the iter function will be executed locally as the entire DSet is read back.

Creates a new dataset whose elements are the results of applying the given function to each of the elements of the dataset. The given function will be applied per collection as the dataset is being distributedly iterated. The entire dataset never materialize entirely.

Create a new DSet whose elements are the results of applying the given function to the corresponding elements of the two DSets pairwise The two DSet must have the same partition mapping structure and same number of element (e.g.), established via Split.

Create a new DSet whose elements are the results of applying the given function to the corresponding elements of the three DSets pairwise

Create a new DSet whose elements are the results of applying the given function to the corresponding elements of the four DSets pairwise

Creates a new dataset whose elements are the results of applying the given function to each collection of the elements of the dataset. In the input DSet, a parttition can have multiple collections, the size of the which is determined by the SerializationLimit of the cluster.

Creates a new dataset whose elements are the results of applying the given function to each of the elements of the dataset. The first integer index passed to the function indicates the partition, and the second integer passed to the function index (from 0) element within the partition

MapReduce, see http://en.wikipedia.org/wiki/MapReduce

MapReduce, see http://en.wikipedia.org/wiki/MapReduce

MapReduce, see http://en.wikipedia.org/wiki/MapReduce

Mixing two DSets that have the same size and partition layout into a single DSet by operating a function on the individual data (row).

Mixing two DSets that have the same size and partition layout into a single DSet by operating a function on the individual data (row).

Mixing three DSets that have the same size and partition layout into a single DSet by operating a function on the individual data (row).

Mixing four DSets that have the same size and partition layout into a single DSet by operating a function on the individual data (row).

Multicast a DSet over network, replicate its content to all peers in the cluster.

Map DSet , in which func is an Task<_> function that may contains asynchronous operation. You will need to start the 1st task in the mapping function. Prajna will not be able to start the task for you as the returned task may not be the a Task in the creation state. see: http://blogs.msdn.com/b/pfxteam/archive/2012/01/14/10256832.aspx

Map DSet , in which func is an Task<_> function that may contains asynchronous operation. The first integer index passed to the function indicates the partition, and the second integer passed to the function index (from 0) element within the partition. You will need to start the 1st task in the mapping function. Prajna will not be able to start the task for you as the returned task may not be the a Task in the creation state. see: http://blogs.msdn.com/b/pfxteam/archive/2012/01/14/10256832.aspx

Read DSet back to local machine and print each value. Caution: the iter function will be executed locally as the entire DSet is read back.

Reduces the elements using the specified 'reducer' function

Reorganization collection in a dataset to accommodate a certain parallel execution degree.

Apply a partition function, repartition elements of dataset across nodes in the cluster. The number of partitions remains unchanged.

Apply a partition function, repartition elements of dataset into 'numPartitions" partitions across nodes in the cluster.

Apply a partition function, repartition elements of dataset across nodes in the cluster according to the setting specified by "param".

merge all rows of a partition into a single collection object

Reorganization collection of a dataset. If numRows = 1, the dataset is split into one row per collection. If numRows = Int32.MaxValue, the data set is merged so that all rows in a partition is grouped into a single collection.

Reorganization collection of a dataset so that each collection has a single row.

Save the DSet to HDD. This is an action.

Save the DSet to HDD using "name". This is an action.

Save the DSet to HDD using "name". This is an action. Attach a monitor function that select elements that will be fetched to local machine to be iterated by 'localIterFunc'

Save the DSet to HDD. This is an action. Attach a monitor function that select elements that will be fetched to local machine to be iterated by 'localIterFunc'

Generate a distributed dataset through a customerized seq functional delegate running on each of the machine. Each node runs a local instance of the functional delegate, which generates a seq('U) that forms one partition of DSet('U). The number of partitions of DSet('U) is N, which is the number of the nodes in the cluster. The NumReplications is 1. If any node in the cluster is not responding, the dataset does not contain the data resultant from the functional delegate in that node.

Generate a distributed dataset through customerized seq functional delegates running on each of the machine. The number of partitions of dataset is numPartitions. If any node in the cluster is not responding, the partition may be rerun at a different node.

Generate a distributed dataset through customerized seq functional delegates running on each of the machine. Each node runs num local instance of the functional delegates, each of which forms one partition of the dataset. The number of partitions of dataset is N * num, where N is the number of the nodes in the cluster, and num is the number of partitions per node. The NumReplications is 1. If any node in the cluster is not responding, the dataset does not contain the data resultant from the functional delegates in that node.

Correlated split a dataset to two, each of which is created by running a functional delegate that maps the element of the original dataset. The resultant datasets all have the same partition and collection structure of the original dataset.

Correlated split a dataset to three, each of which is created by running a functional delegate that maps the element of the original dataset. The resultant datasets all have the same partition and collection structure of the original dataset.

Correlated split a dataset to four, each of which is created by running a functional delegate that maps the element of the original dataset. The resultant datasets all have the same partition and collection structure of the original dataset.

Store a sequence to a persisted DSet

Store a sequence to a persisted DSet

Store a sequence to a persisted DSet

Convert DSet to a sequence seq<'U>

Merge the content of multiple dataset into a single dataset. The original dataset become partitions of the resultant dataset. This can be considered as merge dataset by rows, and all dataset have the same column structure.

Creates a new dataset whose elements are the results of applying the given function to each of the elements of the dataset. The given function is an async function, in which elements within the same collection are parallelly executed with Async.Parallel

Creates a new dataset whose elements are the results of applying the given function to each of the elements of the dataset. The given function is an async function, in which elements within the same collection are parallelly executed with Async.Parallel. The first integer index passed to the function indicates the partition, and the second integer passed to the function index (from 0) element within the partition.

Bin sort the DSet. Apply a partition function, repartition elements of dataset across nodes in the cluster. The number of partitions remains unchanged. Elements within each partition/bin are sorted using the 'comparer'.

Bin sort the DSet. Apply a partition function, repartition elements of dataset into 'numPartitions" partitions across nodes in the cluster. Elements within each partition/bin are sorted using the 'comparer'.

Bin sort the DSet. Apply a partition function, repartition elements of dataset across nodes in the cluster according to the setting specified by "param". Elements within each partition/bin are sorted using the 'comparer'.

Bypass the Dset in 2 ways. One of the DSet is pulled, the other DSet will be in push dataflow.

Bypass the Dset in 2 ways. One of the DSet is pulled, the other DSet will be in push dataflow.

Bypass the Dset in 3 ways. One of the DSet is pulled, the other two DSets will be in push dataflow.

Bypass the Dset in 4 ways. One of the DSet is pulled, the other three DSets will be in push dataflow.

Bypass the Dset in n ways. One of the DSet is pulled, the other n - 1 DSets will be in push dataflow.

Cache DSet content in RAM, in raw object form.

Applies the given function to each element of the dataset. The new dataset comprised of the results for each element where the function returns Some with some value.

Applies the given function to each element of the dataset and concatenates all the results.

Count the number of elements(rows) in the DSet

Cross Join, for each entry in x:Dset<'U> and x1:Dset<'U1> apply mapFunc:'U->'U1->'U2 The resultant set is the product of Dset<'U> and Dset<'U1>

Cross Join with Filtering, for each entry in x:DSet<'U> and x1:DSet<'U1> apply mapFunc:'U->'U1->'U2 option. Filter out all those result that returns None.

Cross Join with Fold, for each entry in x:DSet<'U> and x1:DSet<'U1> apply mapFunc:'U->'U1->'U2, and foldFunc: 'S->'U2->'S

Span a sequence (dataset) to a distributed dataset by splitting the sequence to N partitions, with N being the number of nodes in the cluster. Each node get one partition of the dataset. The number of partitions of the distributed dataset is N, which is the number of the active nodes in the cluster.

Span a sequence (dataset) to a distributed dataset by splitting the sequence to NUM * N partitions, with NUM being the number of partitions on each node, and N being the number of nodes in the cluster. Each node get NUM partition of the dataset.

Execute a distributed customerized functional delegate on each of the machine. The functional delegate is running remotely on each node. If any node in the cluster is not responding, the functional delegate does not run on that node.

Execute num distributed customerized functional delegates on each of the machine. The functional delegate is running remotely on each node. If any node in the cluster is not responding, the functional delegates do not run on that node.

Creates a new dataset containing only the elements of the dataset for which the given predicate returns true.

Fold the entire DSet with a fold function, an aggregation function, and an initial state. The initial state is broadcasted to each partition. Within each partition, the elements are folded into the state variable using 'folder' function. Then 'aggrFunc' is used to aggregate the resulting state variables from all partitions to a single state.

Identity Mapping, the new DSet is the same as the old DSet, with an encode function installed.

Generate a distributed dataset by importing a customerized functional delegate from a local contract store. The imported functional delegate is usually exported by another service (either in the same remote container or in anotherPrajna remote container. Each node runs one local instance of the functional delegate, which forms one partition of the dataset. The number of partitions of dataset is N, where N is the number of the nodes in the cluster. If any node in the cluster is not responding, the dataset does not contain the data resultant from the functional delegates in that node.

Generate a distributed dataset by importing customerized functional delegates from a local contract store. The imported functional delegate is usually exported by another service (either in the same remote container or in anotherPrajna remote container. Each node runs multiple local instance of the functional delegates, each of which forms one partition of the dataset. The number of partitions of dataset is N * num , where N is the number of the nodes in the cluster, and num is the number of contracts in the contract list. If any node in the cluster is not responding, the dataset does not contain the data resultant from the functional delegates in that node.

Create a distributed dataset on the distributed cluster, with each element created by a functional delegate.

Create a distributed dataset on the distributed cluster, with each element created by a functional delegate, using a given number of parallel execution per node. The functional delegate that create each element in the dataset, the integer index passed to the function indicates the partition, and the second integer passed to the function index (from 0) element within the partition. The functional delegate that returns the size of the partition, the integer index passed to the function indicates the partition. The DSet to operate on

Create a distributed dataset on the distributed cluster, with each element created by a functional delegate.

Iterate the given function to each element. This is an action.

Lazily save the DSet to HDD. This is a lazy evaluation. This DSet must be a branch generated by Bypass or a child of such branch. The save action will be triggered when a pull happens on other branches generated by Bypass.

Lazily save the DSet to HDD using "name". This is a lazy evaluation. This DSet must be a branch generated by Bypass or a child of such branch. The save action will be triggered when a pull happens on other branches generated by Bypass.

Trigger to load metadata.

Read DSet back to local machine and apply a function on each value. Caution: the iter function will be executed locally as the entire DSet is read back.

Creates a new dataset whose elements are the results of applying the given function to each of the elements of the dataset. The given function will be applied per collection as the dataset is being distributedly iterated. The entire dataset never materialize entirely.

Create a new DSet whose elements are the results of applying the given function to the corresponding elements of the two DSets pairwise The two DSet must have the same partition mapping structure and same number of element (e.g.), established via Split.

Create a new DSet whose elements are the results of applying the given function to the corresponding elements of the three DSets pairwise

Create a new DSet whose elements are the results of applying the given function to the corresponding elements of the four DSets pairwise

Creates a new dataset whose elements are the results of applying the given function to each collection of the elements of the dataset. In the input DSet, a parttition can have multiple collections, the size of the which is determined by the SerializationLimit of the cluster.

Creates a new dataset whose elements are the results of applying the given function to each of the elements of the dataset. The first integer index passed to the function indicates the partition, and the second integer passed to the function index (from 0) element within the partition

MapReduce, see http://en.wikipedia.org/wiki/MapReduce

MapReduce, see http://en.wikipedia.org/wiki/MapReduce

MapReduce, see http://en.wikipedia.org/wiki/MapReduce

Mixing two DSets that have the same size and partition layout into a single DSet by operating a function on the individual data (row).

Mixing two DSets that have the same size and partition layout into a single DSet by operating a function on the individual data (row).

Mixing three DSets that have the same size and partition layout into a single DSet by operating a function on the individual data (row).

Mixing four DSets that have the same size and partition layout into a single DSet by operating a function on the individual data (row).

Multicast a DSet over network, replicate its content to all peers in the cluster.

Map DSet , in which func is an Task<_> function that may contains asynchronous operation. You will need to start the 1st task in the mapping function. Prajna will not be able to start the task for you as the returned task may not be the a Task in the creation state. see: http://blogs.msdn.com/b/pfxteam/archive/2012/01/14/10256832.aspx

Map DSet , in which func is an Task<_> function that may contains asynchronous operation. The first integer index passed to the function indicates the partition, and the second integer passed to the function index (from 0) element within the partition. You will need to start the 1st task in the mapping function. Prajna will not be able to start the task for you as the returned task may not be the a Task in the creation state. see: http://blogs.msdn.com/b/pfxteam/archive/2012/01/14/10256832.aspx

Read DSet back to local machine and apply a function on each value. Caution: the iter function will be executed locally as the entire DSet is read back.

Reduces the elements using the specified 'reducer' function

Reorganization collection in a dataset to accommodate a certain parallel execution degree.

Apply a partition function, repartition elements of dataset across nodes in the cluster. The number of partitions remains unchanged.

Apply a partition function, repartition elements of dataset into 'numPartitions" partitions across nodes in the cluster.

Apply a partition function, repartition elements of dataset across nodes in the cluster according to the setting specified by "param".

merge all rows of a partition into a single collection object

Reorganization collection of a dataset. If numRows = 1, the dataset is split into one row per collection. If numRows = Int32.MaxValue, the data set is merged so that all rows in a partition is grouped into a single collection.

Reorganization collection of a dataset so that each collection has a single row.

Save the DSet to HDD. This is an action.

Save the DSet to HDD using "name". This is an action.

Save the DSet to HDD using "name". This is an action. Attach a monitor function that select elements that will be fetched to local machine to be iterated by 'localIterFunc'

Save the DSet to HDD. This is an action. Attach a monitor function that select elements that will be fetched to local machine to be iterated by 'localIterFunc'

Generate a distributed dataset through a customerized seq functional delegate running on each of the machine. Each node runs a local instance of the functional delegate, which generates a seq('U) that forms one partition of DSet('U). The number of partitions of DSet('U) is N, which is the number of the nodes in the cluster. The NumReplications is 1. If any node in the cluster is not responding, the dataset does not contain the data resultant from the functional delegate in that node.

Generate a distributed dataset through customerized seq functional delegates running on each of the machine. The number of partitions of dataset is numPartitions. If any node in the cluster is not responding, the partition may be rerun at a different node.

Generate a distributed dataset through customerized seq functional delegates running on each of the machine. Each node runs num local instance of the functional delegates, each of which forms one partition of the dataset. The number of partitions of dataset is N * num, where N is the number of the nodes in the cluster, and num is the number of partitions per node. The NumReplications is 1. If any node in the cluster is not responding, the dataset does not contain the data resultant from the functional delegates in that node.

Correlated split a dataset to two, each of which is created by running a functional delegate that maps the element of the original dataset. The resultant datasets all have the same partition and collection structure of the original dataset.

Correlated split a dataset to three, each of which is created by running a functional delegate that maps the element of the original dataset. The resultant datasets all have the same partition and collection structure of the original dataset.

Correlated split a dataset to four, each of which is created by running a functional delegate that maps the element of the original dataset. The resultant datasets all have the same partition and collection structure of the original dataset. They can be combined later by Map4 transforms.

Store a sequence to a persisted DSet

Convert DSet to a sequence seq<'U>

Find dataset with name that matches the search pattern.

Merge the content of multiple dataset into a single dataset. The original dataset become partitions of the resultant dataset. This can be considered as merge dataset by rows, and all dataset have the same column structure.